PubMed Health⌕ Search
Home/Archive/PubMed

PubMed · 8697228

A numerical study on learning curves in stochastic multilayer feedforward networks.

K R Müller·M Finke·N Murata·K Schulten·S Amari
View original source ↗Export metadata ↓

Abstract

The universal asymptotic scaling laws proposed by Amari et al. are studied in large scale simulations using a CM5. Small stochastic multilayer feedforward networks trained with backpropagation are investigated. In the range of a large number of training patterns t, the asymptotic generalization error scales as 1/t as predicted. For a medium range t a faster 1/t2 scaling is observed. This effect is explained by using higher order corrections of the likelihood expansion. It is shown for small t that the scaling law changes drastically, when the network undergoes a transition from strong overfitting to effective learning.

Explore related subjects

LearningNeural Networks, ComputerNeuronsJournal ArticleResearch Support, Non-U.S. Gov'tResearch Support, U.S. Gov't, P.H.S.

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

K R Müller, M Finke, N Murata, K Schulten, S Amari. 1996-07-01. A numerical study on learning curves in stochastic multilayer feedforward networks.. https://doi.org/10.1162/neco.1996.8.5.1085

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2006

A statistical property of multiagent learning based on Markov decision process.

We exhibit an important property called the asymptotic equipartition property (AEP) on empirical sequences in an ergodic multiagent Markov decision process (MDP). Using the AEP which facilitates the analysis of multiagent learning, we give a statistical property of multiagent learning, such as reinforcement learning (RL), near the end of the learning process. We examine the effect of the conditions among the agents on the achievement of a cooperative policy in three different cases: blind, visible, and communicable. Also, we derive a bound on the speed with which the empirical sequence converges to the best sequence in probability, so that the multiagent learning yields the best cooperative result.

Learning↗
PubMed2006

Second order neurons and learning in Cohen-Grossberg networks.

The well known Cohen-Grossberg network is modified to include second order neural interconnections and also to have a learning component. Sufficient conditions are obtained for the existence of a globally exponentially stable equilibrium. The model provides a two-fold generalization of the Cohen-Grossberg network in the sense if one removes the learning component, then one gets a network with second order synaptic interactions; if both the learning component and the second order interactions are removed, then the model reduces to the standard Cohen-Grossberg network.

Learning↗