PubMed Health⌕ Search

Biomedical subjects

O Winther

Publications and source records attributed to O Winther.

18 recordsLinked to original sources
PubMed2001

Adaptive and self-averaging Thouless-Anderson-Palmer mean-field theory for probabilistic modeling.

We develop a generalization of the Thouless-Anderson-Palmer (TAP) mean-field approach of disorder physics, which makes the method applicable to the computation of approximate averages in probabilistic models for real data. In contrast to the conventional TAP approach, where the knowledge of the distribution of couplings between the random variables is required, our method adapts to the concrete set of couplings. We show the significance of the approach in two ways: Our approach reproduces replica symmetric results for a wide class of toy models (assuming a nonglassy phase) with given disorder distributions in the thermodynamic limit. On the other hand, simulations on a real data model demonstrate that the method achieves more accurate predictions as compared to conventional TAP approaches.

Journal Article↗
PubMed2001

Tractable approximations for probabilistic models: the adaptive Thouless-Anderson-Palmer mean field approach.

We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the Thouless-Anderson-Palmer (TAP) approach of disorder physics. In contrast to conventional TAP, where the knowledge of the distribution of couplings between the random variables is required, our method adapts to the concrete couplings. We demonstrate the validity of our approach, which is so far restricted to models with nonglassy behavior, by replica calculations for a wide class of models as well as by simulations for a real data set.

Journal Article↗
PubMed2000

Gaussian processes for classification: mean-field algorithms.

We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler "naive" mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.

Algorithms↗
PubMed1993

A quantitative study of pruning by optimal brain damage.

The optimal brain damage (OBD) scheme of Le Cun, Denker and Solla for pruning of feedforward networks has been implemented and applied to the contiguity classification problem. It is shown that OBD improves the learning curve (the test error as a function of the number of examples). By inspecting the architectures obtained through pruning, it is found that the networks with less parameters have the smallest test error in agreement with "Ockhams Razor". Based on this, we propose a heuristic which selects the smallest successful architecture among a group of pruned networks and we show that it leads to very efficient optimization of the architecture. The validity of the approximations involved in OBD are discussed and it is found that they are surprisingly accurate for the problem studied.

Animals↗