PubMed · 12662506
Network Synthesis through Data-Driven Growth and Decay.
Abstract
AN ALGORITHM FOR ADDITION AND DELETION (ADDEL) OF RESOURCES DURING LEARNING IS DEVELOPED TO ACHIEVE TWO GOALS: (1) to find feed-forward multilayer networks that are as small as possible, (2) to find an appropriate structure for such small networks. These goals are accomplished by operating alternately between an adding phase and a deleting phase while learning the given input-output associations. The adding phase develops a crude structure by filling in resources (connections, units and layers) at a virtual multilayer network with a maximum of L possible layers. The deleting phase then removes any unnecessary connections to obtain a refined structure. The additions and deletions are done based on a sensitivity measure and a corresponding probability rule so that only the synapses which are most effective in reducing the output error are preserved. A generalization error estimated from a validation set is used to control the alternation between the two learning phases and the termination of learning. Simulations, including handwritten digit recognition, demonstrate that the algorithm is effective in finding an appropriate network structure for a small network which can generalize well. The algorithm is used to investigate when the size of a network is important for generalization. Copyright 1997 Elsevier Science Ltd.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Demetri Psaltis, Chuanyi Ji. 1997. Network Synthesis through Data-Driven Growth and Decay.. https://doi.org/10.1016/s0893-6080(97)00032-4. https://pubmed.health/record/pubmed%3A12662506/network-synthesis-through-data-driven-growth-and-decay
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite PubMed Health by Indiciae: BibTeX · RIS.