Optimal nonlinear training in the multi-class proximity problem.

D Bollé; G Jongen; G M Shim

doi:10.1142/s0129065796000634

Home/Archive/PubMed

PubMed · 9040066

Optimal nonlinear training in the multi-class proximity problem.

D Bollé·G Jongen·G M Shim

View original source ↗Export metadata ↓

Abstract

Using a signal-to-noise analysis, the effects of nonlinear modulation of the Hebbian learning rule in the multi-class proximity problem are investigated. Both random classification and classification provided by a Gaussian and a binary teacher are treated. Analytic expressions are derived for the learning and generalization rates around an old and a new prototype. For the proximity problem with binary inputs but Q'-state outputs, it is shown that the optimal modulation is a combination of a hyperbolic tangent and a linear function. As an illustration, numerical results are presented for the two-class and the Q' = 3 multi-class problem.

Explore related subjects

Learning Neural Networks, Computer Nonlinear Dynamics Journal Article Research Support, Non-U.S. Gov't

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

D Bollé, G Jongen, G M Shim. 1996. Optimal nonlinear training in the multi-class proximity problem.. https://doi.org/10.1142/s0129065796000634

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed1998

Complexity issues in natural gradient descent method for training multilayer perceptrons.

The natural gradient descent method is applied to train an n-m-1 multilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).

Learning↗

PubMed1998

Online learning from finite training sets and robustness to input bias.

We analyze online gradient descent learning from finite training sets at noninfinitesimal learning rates eta. Exact results are obtained for the time-dependent generalization error of a simple model system: a linear network with a large number of weights N, trained on p = alphaN examples. This allows us to study in detail the effects of finite training set size alpha on, for example, the optimal choice of learning rate eta. We also compare online and offline learning, for respective optimal settings of eta at given final learning time. Online learning turns out to be much more robust to input bias and actually outperforms offline learning when such bias is present; for unbiased inputs, online and offline learning perform almost equally well.

Learning↗

PubMed1997

A mathematical model of neural information processing at the cellular level.

The basis for this neuronal model is that the properties of excitable membranes are controlled by biochemical reactions occurring in the nerve cells. The kinetics of these supposed chemical reactions is described by a set of first order differential equations. We considered the effect of regulation of the properties of sodium channels. This allows the simulation of the changes in the neuron's electrical activity parameters occurring during learning, associated with its excitability. The neuronal model exhibits different excitability after the learning procedure relative to the different input signals that corresponds to the experimental data.

Learning↗