SEARCH · PubMed Health

Results for “Shannon information entropy”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 recordsLinked to original sources

PubMed2005

Characteristic features of Shannon information entropy of confined atoms.

The Shannon information entropy of 1-normalized electron density in position and momentum space Sr and Sp, and the sum ST, respectively, are reported for the ground-state H, He+, Li2+, H-, He, Li+, Li, and B atoms confined inside an impenetrable spherical boundary defined by radius R. We find new characteristic features in ST denoted by well-defined minimum and maximum as a function of confinement. The results are analyzed in the background of the irreducible lower bound stipulated by the entropy uncertainty principle [I. Bialynicki-Birula and J. Mycielski, Commun. Math. Phys. 44, 129 (1975)]. The spherical confinement model leads to the ST values which satisfy the lower bound up to the limits of extreme confinements with the interesting new result displaying regions over which a set of upper and lower bounds to the information entropy sum can be locally prescribed. Similar calculations on the H atom in 2s excited states are presented and their novel characteristics are discussed.

Journal Article↗

PubMed2026

Research on multi-trait genome association study method based on Shannon information entropy.

BACKGROUND: Genetic analysis of complex traits is crucial for elucidating disease mechanisms and biological inheritance processes. However, traditional Genome-wide Association Study (GWAS) for single trait often fail to capture the synergistic effects of genetic loci on multiple traits. METHODS: This study proposes a method for analyzing the association between multiple traits and gene regions based on Shannon information entropy. Innovatively, Shannon information entropy is introduced to integrate gene region information as genetic entropy, thereby constructing an Inverse Shannon Entropy-Multi-Trait Association Analysis of Gene Region genetic model (InvSE-MTAGR). Furthermore, a partial regression test is applied to the model to establish the Inverse Partial Shannon Entropy-Multi-Trait Association Analysis of Gene Region method (InvPSE-MTAGR). When performing multi-trait analysis with InvSE-MTAGR, the method achieved statistical significance by accumulating minor effects, thereby enhancing the ability to identify pleiotropic gene regions. RESULTS: The simulation results showed that the proposed multi-trait gene region association analysis method performed well in terms of both Type I error rate control and statistical power. Leveraging tomato and sorghum datasets for validation, the proposed multi-trait gene region association analysis method based on Shannon information entropy accurately pinpointed most of the gene regions harboring candidate genes. CONCLUSION: The study reveals the advantage of multi-trait method in integrating weak-effect pleiotropic signals and capturing the correlation among traits, which provides an efficient theoretical tool for dynamic analysis of complex multi-trait genetic networks and multi-target collaborative breeding of crops.

Genome-Wide Association Study↗

PubMed2004

Finite size scaling for the atomic Shannon-information entropy.

We have developed the finite size scaling method to treat the criticality of Shannon-information entropy for any given quantum Hamiltonian. This approach gives very accurate results for the critical parameters by using a systematic expansion in a finite basis set. To illustrate this approach we present a study to estimate the critical exponents of the Shannon-information entropy S approximately (lambda-lambda(c))(alpha(S) ), the electronic energy E approximately (lambda-lambda(c))(alpha(E) ), and the correlation length xi approximately mid R:lambda-lambda(c)mid R:(-nu) for atoms with the variable lambda=1/Z, which is the inverse of the nuclear charge Z. This was realized by approximating the multielectron atomic Hamiltonian with a one-electron model Hamiltonian. This model is very accurate for describing the electronic structure of the atoms near their critical points. For several atoms in their ground electronic states, we have found that the critical exponents (alpha(E),nu,alpha(S)) for He (Z=2), C (Z=6), N (Z=7), F (Z=9), and Ne (Z=10), respectively, are (1, 0, 0). At the critical points lambda(c)=1/Z(c), the bound state energies become absorbed or degenerate with continuum states and the entropies reach their maximum values, indicating a maximal delocalization of the electronic wave function.

Journal Article↗

PubMed1996

The Shannon information entropy of protein sequences.

A comprehensive data base is analyzed to determine the Shannon information content of a protein sequence. This information entropy is estimated by three methods: a k-tuplet analysis, a generalized Zipf analysis, and a "Chou-Fasman gambler." The k-tuplet analysis is a "letter" analysis, based on conditional sequence probabilities. The generalized Zipf analysis demonstrates the statistical linguistic qualities of protein sequences and uses the "word" frequency to determine the Shannon entropy. The Zipf analysis and k-tuplet analysis give Shannon entropies of approximately 2.5 bits/amino acid. This entropy is much smaller than the value of 4.18 bits/amino acid obtained from the nonuniform composition of amino acids in proteins. The "Chou-Fasman" gambler is an algorithm based on the Chou-Fasman rules for protein structure. It uses both sequence and secondary structure information to guess at the number of possible amino acids that could appropriately substitute into a sequence. As in the case for the English language, the gambler algorithm gives significantly lower entropies than the k-tuplet analysis. Using these entropies, the number of most probable protein sequences can be calculated. The number of most probable protein sequences is much less than the number of possible sequences but is still much larger than the number of sequences thought to have existed throughout evolution. Implications of these results for mutagenesis experiments are discussed.

Algorithms↗

PubMed2005

Information entropy, information distances, and complexity in atoms.

Shannon information entropies in position and momentum spaces and their sum are calculated as functions of Z(2 < or = Z < or = 54) in atoms. Roothaan-Hartree-Fock electron wave functions are used. The universal property S = a + b ln Z is verified. In addition, we calculate the Kullback-Leibler relative entropy, the Jensen-Shannon divergence, Onicescu's information energy, and a complexity measure recently proposed. Shell effects at closed-shell atoms are observed. The complexity measure shows local minima at the closed-shell atoms indicating that for the above atoms complexity decreases with respect to neighboring atoms. It is seen that complexity fluctuates around an average value, indicating that the atom cannot grow in complexity as Z increases. Onicescu's information energy is correlated with the ionization potential. Kullback distance and Jensen-Shannon distance are employed to compare Roothaan-Hartree-Fock density distributions with other densities of previous works.

Journal Article↗

PubMed2002

Nonextensive entropies derived from form invariance of pseudoadditivity.

The form invariance of pseudoadditivity is shown to determine the structure of nonextensive entropies. Nonextensive entropy is defined as the appropriate expectation value of nonextensive information content, similar to the definition of Shannon entropy. Information content in a nonextensive system is obtained uniquely from generalized axioms by replacing the usual additivity with pseudoadditivity. The satisfaction of the form invariance of the pseudoadditivity of nonextensive entropy and its information content is found to require the normalization of nonextensive entropies. The proposed principle requires the same normalization as that derived previously [A.K. Rajagopal and S. Abe, Phys. Rev. Lett. 83, 1711 (1999)], but is simpler and establishes a basis for the systematic definition of various entropies in nonextensive systems.

Journal Article↗

PubMed2006

Information entropies for eigendensities of homogeneous potentials.

For homogeneous potentials, the sum S(T), of position and momentum Shannon information entropies Sr and Sp is shown to be independent of the coupling strength scaling. The other commonly used uncertainty like products also follow similar behavior. The ramifications of this scaling property in the cases of hydrogenlike, harmonic oscillator, Morse, and Poeschl-Teller potentials are discussed with the example of S(T).

Journal Article↗

PubMed2005

Protein sequence entropy is closely related to packing density and hydrophobicity.

We investigated the correlation between the Shannon information entropy, 'sequence entropy', with respect to the local flexibility of native globular proteins as described by inverse packing density. These are determined at each residue position for a total set of 130 query proteins, where sequence entropies are calculated from each set of aligned residues. For the accompanying aggregate set of 130 alignments, a strong linear correlation is observed between the calculated sequence entropy and the corresponding inverse packing density determined at an associated residue position. This region of linearity spans the range of C(alpha) packing densities from 12 to 25 amino acids within a sphere of 9 angstrom radius. Three different hydrophobicity scales all mimic the behavior of the sequence entropies. This confirms the idea that the ability to accommodate mutations is strongly dependent on the available space and on the propensity for each amino acid type to be buried. Future applications of these types of methods may prove useful in identifying both core and flexible residues within a protein.

Entropy↗

PubMed2000

Stochastic dynamics of time correlation in complex systems with discrete time

In this paper we present the concept of description of random processes in complex systems with discrete time. It involves the description of kinetics of discrete processes by means of the chain of finite-difference non-Markov equations for time correlation functions (TCFs). We have introduced the dynamic (time dependent) information Shannon entropy S(i)(t) where i=0,1,2,3,ellipsis, as an information measure of stochastic dynamics of time correlation (i=0) and time memory (i=1,2,3,ellipsis). The set of functions S(i)(t) constitute the quantitative measure of time correlation disorder (i=0) and time memory disorder (i=1,2,3,ellipsis) in complex system. The theory developed started from the careful analysis of time correlation involving dynamics of vectors set of various chaotic states. We examine two stochastic processes involving the creation and annihilation of time correlation (or time memory) in details. We carry out the analysis of vectors' dynamics employing finite-difference equations for random variables and the evolution operator describing their natural motion. The existence of TCF results in the construction of the set of projection operators by the usage of scalar product operation. Harnessing the infinite set of orthogonal dynamic random variables on a basis of Gram-Shmidt orthogonalization procedure tends to creation of infinite chain of finite-difference non-Markov kinetic equations for discrete TCFs and memory functions (MFs). The solution of the equations above thereof brings to the recurrence relations between the TCF and MF of senior and junior orders. This offers new opportunities for detecting the frequency spectra of power of entropy function S(i)(t) for time correlation (i=0) and time memory (i=1,2,3,ellipsis). The results obtained offer considerable scope for attack on stochastic dynamics of discrete random processes in a complex systems. Application of this technique on the analysis of stochastic dynamics of RR intervals from human ECG's shows convincing evidence for a non-Markovian phenomemena associated with a peculiarities in short- and long-range scaling. This method may be of use in distinguishing healthy from pathologic data sets based in differences in these non-Markovian properties.

Journal Article↗

PubMed1999

Discretized torsional dynamics and the folding of an RNA chain.

The aim of this work is to implement a discrete coarse codification of local torsional states of the RNA chain backbone in order to explore the long-time limit dynamics and ultimately obtain a coarse solution to the RNA folding problem. A discrete representation of the soft-mode dynamics is turned into an algorithm for a rough structure prediction. The algorithm itself is inherently parallel, as it evaluates concurrent folding possibilities by pattern recognition, but it may be implemented in a personal computer as a chain of perturbation-translation-renormalization cycles performed on a binary matrix of local topological constraints. This requires suitable representational tools and a periodic quenching of the dynamics for system renormalization. A binary coding of local topological constraints associated with each structural motif is introduced, with each local topological constraint corresponding to a local torsional state. This treatment enables us to adopt a computation time step far larger than hydrodynamic drag time scales. Accordingly, the solvent is no longer treated as a hydrodynamic drag medium. Instead we incorporate its capacity for forming local conformation-dependent dielectric domains. Each translation of the matrix of local topological constraints (LTM's) depends on the conformation-dependent local dielectric created by a confined solvent. Folding pathways are resolved as transitions between patterns of locally encoded structural signals which change within the 1 ns-100 ms time scale range. These coarse folding pathways are generated by a search at regular intervals for structural patterns in the LTM. Each pattern is recorded as a base-pairing pattern (BPP) matrix, a consensus-evaluation operation subject to a renormalization feedback loop. Since several mutually conflicting consensus evaluations might occur at a given time, the need arises for a probabilistic approach appropriate for an ensemble of RNA molecules. Thus, a statistical dynamics of consensus formation is determined by the time evolution of the base pairing probability matrix. These dynamics are generated for a functional RNA molecule, a representative of the so-called group I ribozymes, in order to test the model. The resulting ensemble of conformations is sharply peaked and the most probable structure features the predominance of all phylogenetically conserved intrachain helices tantamount to ribozyme function. Furthermore, the magnesium-aided cooperativity that leads to the shaping of the catalytic core is elucidated. Once the predictive folding algorithm has been implemented, the validity of the so-called "adiabatic approximation" is tested. This approximation requires that conformational microstates be lumped up into BPP's which are treated as quasiequilibrium states, while folding pathways are coarsely represented as sequences of BPP transitions. To test the validity of this adiabatic ansatz, a computation of the coarse Shannon information entropy sigma associated to the specific partition of conformation space into BPP's is performed taking into account the LTM evolution and contrasted with the adiabatic computation. The results reveal a subordination of torsional microstate dynamics to BPP transitions within time scales relevant to folding. This adiabatic entrainment in the long-time limit is thus identified as responsible for the expediency of the folding process.

Algorithms↗

PubMed2005

Proposed mechanism for learning and memory erasure in a white-noise-driven sleeping cortex.

Understanding the structure and purpose of sleep remains one of the grand challenges of neurobiology. Here we use a mean-field linearized theory of the sleeping cortex to derive statistics for synaptic learning and memory erasure. The growth in correlated low-frequency high-amplitude voltage fluctuations during slow-wave sleep (SWS) is characterized by a probability density function that becomes broader and shallower as the transition into rapid-eye-movement (REM) sleep is approached. At transition, the Shannon information entropy of the fluctuations is maximized. If we assume Hebbian-learning rules apply to the cortex, then its correlated response to white-noise stimulation during SWS provides a natural mechanism for a synaptic weight change that will tend to shut down reverberant neural activity. In contrast, during REM sleep the weights will evolve in a direction that encourages excitatory activity. These entropy and weight-change predictions lead us to identify the final portion of deep SWS that occurs immediately prior to transition into REM sleep as a time of enhanced erasure of labile memory. We draw a link between the sleeping cortex and Landauer's dissipation theorem for irreversible computing [R. Landauer, IBM J. Res. Devel. 5, 183 (1961)], arguing that because information erasure is an irreversible computation, there is an inherent entropy cost as the cortex transits from SWS into REM sleep.

Action Potentials↗

PubMed2003

Shannon entropy as an indicator of atomic avoided crossings in strong parallel magnetic and electric fields.

Avoided crossings are the most distinctive atomic spectroscopic features in the presence of magnetic and electric fields. We point out the role of Shannon's information entropy as an indicator or predictor of these phenomena by studying the dynamics of some excited states of hydrogen in the presence of parallel magnetic and electric fields. Moreover, in addition to the well-known energy level repulsion, it is found that Shannon's entropy manifests the informational exchange of the involved states as the magnetic field strength is varied across the narrow region where an avoided crossing occurs.

Journal Article↗

PubMed2006

The information content of high-frequency environmental monitoring data signals pollution events in the coastal ocean.

There are an increasing number of coastal ocean observing systems that deploy new technology for environmental sensing and stream these data in near-real-time to end-users (e.g., scientists and coastal managers) via the worldwide web. The temporal resolution, spatial coverage, and accessibility of these data open up new opportunities for better understanding and managing the coastal ocean, but they also present enormous challenges relative to data processing and data interpretation, particularly in cases where these data are to inform rapid management decision making. Here we demonstrate that changes in surf zone water quality at a popular beach in southern California are signaled by changes in the Fisher Information and Shannon Entropy of high frequency (1/4 min(-1)) measurements of salinity and temperature in the surf zone. These results support the hypothesis that the information content of environmental signals, such as salinity and temperature, can be used to identify changes in the water quality of the coastal ocean. More generally, the approach described here-of using information theory indices calculated from monitoring data as real-time indicators of environmental change-is quite general, and may therefore be applicable to other situations where rapid management decisions are based on high-frequency measurements of environmental parameters.

Algorithms↗

PubMed2004

Cluster and information entropy patterns in immunoglobulin complementarity determining regions.

Previous studies of antibody binding domains have established many crucial features that include important structural positions, canonical formations, and the geometric correlations with the binding site nature and topography. In this work, position-specific frequency and hierarchical clustering analysis are used to explore the statistical pattern of the residues in the complementarity determining regions of human antibodies. In addition, Shannon's information entropy is computed for the entire heavy and light chains and compared with germline patterns to seek variability due to antibody clonal selection. Results are compared with reported analyses based on structural data and ligand-protein contact point computations based on Protein Data Bank records. Observations derived from the present sequence analysis are consistent with previous structural based methods. In the absence of structural data, methods used in this work can be effective and efficient computational tools used for identifying residues that are important for antigen targeting and predicting the probable amino acid distribution expected at these positions. The results in turn can be applied to help design or plan mutagenesis experiments to improve the binding properties of antibodies.

Amino Acid Sequence↗

PubMed2002

Entropy of protein sequences: an integral approach.

Several classifications of protein spatial structures and their structural elements are known. This makes revealing of the relation between these structural elements and sequence fragments rather topical. The most important move in this direction would be the determination of positional sensitivity levels and ranges between the residues in protein sequences. In this work the Shannon-Weaver informational entropy was used as a disorder criterion for solving this problem. This entropy was computed as function of the distance between the amino acid residues in different sets of unhomological protein sequences. Similarity of this function for different sets of protein sequences was shown. Analysis of informational entropy allows detecting a long-range positional correlation (> or =30) between the amino acid residues and oscillations with periods of 3.6 and 2.9. These oscillation periods correspond to periodicity of alpha- and 3(10)-helices.

Computational Biology↗

PubMed2005

Shannon information, LMC complexity and Rényi entropies: a straightforward approach.

The LMC complexity, an indicator of complexity based on a probabilistic description, is revisited. A straightforward approach allows us to establish the time evolution of this indicator in a near-equilibrium situation and gives us a new insight for interpreting the LMC complexity for a general non equilibrium system. Its relationship with the Rényi entropies is also explained. One of the advantages of this indicator is that its calculation does not require a considerable computational effort in many cases of physical and biological interest.

Entropy↗

PubMed2004

The 'weighted sum of relative entropy': a new index for synonymous codon usage bias.

Shannon entropy from information theory has been applied to estimate the degree of deviation from equal usage of synonymous codons; however, previous attempts have failed to take into account all three aspects of amino acid usage, i.e. (i) the number of distinct amino acids, (ii) their relative frequencies, and (iii) their degree of codon degeneracy. A new index taking into account all of these aspects is proposed. The index, designated as the 'weighted sum of relative entropy' (E(w)), is defined as the sum of the relative entropy of each amino acid weighted by its relative frequency in the sequence. In this paper, we demonstrate that E(w) allows us to avoid some amino acid usage biases and can yield results contradictory to those obtained by previous methods.

Algorithms↗

PubMed2000

Algorithmic complexity in the minority game

In this paper, we present our approach for the study of the complexity of Minority Game using tools from thermodynamics and statistical physics. Previous attempts were based on the behavior of volatility, an observable of the financial markets. Our approach focuses on some properties of the binary stream of outcomes of the game. Physical complexity, a magnitude rooted in Kolmogorov-Chaitin theory, allows us to explain some properties of collective behavior of the agents. Mutual information function, a measure related to Shannon's information entropy, was useful to observe a kind of phase transition when applied to the binary string of the whole history of the game.

Journal Article↗