PubMed · 8697228
A numerical study on learning curves in stochastic multilayer feedforward networks.
Abstract
The universal asymptotic scaling laws proposed by Amari et al. are studied in large scale simulations using a CM5. Small stochastic multilayer feedforward networks trained with backpropagation are investigated. In the range of a large number of training patterns t, the asymptotic generalization error scales as 1/t as predicted. For a medium range t a faster 1/t2 scaling is observed. This effect is explained by using higher order corrections of the likelihood expansion. It is shown for small t that the scaling law changes drastically, when the network undergoes a transition from strong overfitting to effective learning.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
K R Müller, M Finke, N Murata, K Schulten, S Amari. 1996-07-01. A numerical study on learning curves in stochastic multilayer feedforward networks.. https://doi.org/10.1162/neco.1996.8.5.1085. https://pubmed.health/record/pubmed%3A8697228/a-numerical-study-on-learning-curves-in-stochastic-multilayer-feedforward-networks
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite PubMed Health by Indiciae: BibTeX · RIS.