Extensions to minimum relative entropy inversion for noisy data.

Tadeusz J Ulrych; Allan D Woodbury

doi:10.1016/s0169-7722(03)00093-7

Home/Archive/PubMed

PubMed · 14607467

Extensions to minimum relative entropy inversion for noisy data.

Tadeusz J Ulrych·Allan D Woodbury

View original source ↗Export metadata ↓

Abstract

Minimum relative entropy (MRE) and Tikhonov regularization (TR) were compared by Neupauer et al. [Water Resour. Res. 36 (2000) 2469] on the basis of an example plume source reconstruction problem originally proposed by Skaggs and Kabala [Water Resour. Res. 30 (1994) 71] and a boxcar-like function. Although Neupauer et al. [Water Resour. Res. 36 (2000) 2469] were careful in their conclusions to note the basis of these comparisons, we show that TR does not perform well on problems in which delta-like sources are convolved with diffuse-groundwater contamination response functions, particularly in the presence of noise. We also show that it is relatively easy to estimate an appropriate value for epsilon, the hyperparameter needed in the minimum relative entropy solution for the inverse problem in the presence of noise. This can be estimated in a variety of ways, including estimation from the data themselves, analysis of data residuals, and a rigorous approach using the real cepstrum and the Akaike Information Criterion (AIC). Regardless of the approach chosen, for the sample problem reported herein, excellent resolution of multiple delta-like spikes is produced from MRE from noisy, diffuse data. The usefulness of MRE for noisy inverse problems has been demonstrated.

Explore related subjects

Entropy Models, Statistical Reproducibility of Results Water Pollutants Water Supply Comparative Study Journal Article Research Support, Non-U.S. Gov't

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Tadeusz J Ulrych, Allan D Woodbury. 2003. Extensions to minimum relative entropy inversion for noisy data.. https://doi.org/10.1016/s0169-7722(03)00093-7

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2026

Quantifying the peripheral surface information entropy from conformational ensembles of globular protein-peptide complexes.

Predicting favorable protein-peptide binding events remains a central challenge in biophysics, with continued uncertainty surrounding how nonlocal effects shape the global energy landscape. Here, we introduce peripheral surface information entropy, SΨ, a quantitative measure of the statistical variability in apolar and charged non-interacting surface (NIS) proportions across conformational ensembles. Within the Gibbs free-energy relation ΔG = ΔH - TΔS, SΨ is proposed as a computationally tractable entropic proxy rather than a direct thermodynamic observable or stand-alone estimator of binding affinity. Using energy-directed molecular docking via HADDOCK3 and explicit-solvent molecular dynamics simulations, it is demonstrated that favorable binding partners exhibit emergent, low-entropy N-states (discrete macrostates in NIS state space) indicative of preferential apolar/charged surface configurations. Across dozens of peptides and multiple receptor systems (WW, PDZ, and MDM2 domains), dominant N-states persisted under varied docking parameters and initial conditions. A meta-ensemble of 657 complexes from 36 experiments over 15 years confirmed the presence of dominant NIS modes independent of in silico methodology, suggesting an evolutionary selection pressure toward specific NIS fingerprints. These findings establish SΨ as a thermoinformatic descriptor that encodes favorable binding constraints into unique statistical signatures of the NIS.

Entropy↗

PubMed2007

Effects of entropy on the gas-phase pyrolysis of ethyl N,N-dimethylcarbamate.

In this study, we examined the gas-phase pyrolysis of ethyl N,N-dimethylcarbamate theoretically at various theoretical levels. The reaction consists of a two-step mechanism, with N,N-dimethylcarbamic acid and ethylene as reaction intermediates. In the first step, the reaction proceeds via a six-membered cyclic transition state (TS), which is more favorable than that via a four-membered cyclic TS. Here, the contribution of entropy to the overall potential energy surface was found to play an important role in determining the rate-limiting step, which was found to be the second step when viewed in terms of the enthalpy of activation (DeltaH(not equal)), but the first step when entropy changes (-TDeltaS(not equal)) were considered. These results are consistent with experimental findings. Moreover, the experimental activation entropy can be reproduced by using the hindered rotor approximation, which converts some low vibration frequencies that correspond to internal rotational modes into hindered rotors.

Entropy↗

PubMed2007

Analysis and design of echo state networks.

The design of echo state network (ESN) parameters relies on the selection of the maximum eigenvalue of the linearized system around zero (spectral radius). However, this procedure does not quantify in a systematic manner the performance of the ESN in terms of approximation error. This article presents a functional space approximation framework to better understand the operation of ESNs and proposes an information-theoretic metric, the average entropy of echo states, to assess the richness of the ESN dynamics. Furthermore, it provides an interpretation of the ESN dynamics rooted in system theory as families of coupled linearized systems whose poles move according to the input signal dynamics. With this interpretation, a design methodology for functional approximation is put forward where ESNs are designed with uniform pole distributions covering the frequency spectrum to abide by the richness metric, irrespective of the spectral radius. A single bias parameter at the ESN input, adapted with the modeling error, configures the ESN spectral radius to the input-output joint space. Function approximation examples compare the proposed design methodology versus the conventional design.

Entropy↗