Collective computational properties of neural networks: New learning mechanisms.
Explore the source record for details and available documents.
Journal Article↗
SEARCH · PubMed Health
Results for “Neural Networks, Computer”
Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Topology and computational performance of attractor neural networks.
To explore the relation between network structure and function, we studied the computational performance of Hopfield-type attractor neural nets with regular lattice, random, small-world, and scale-free topologies. The random configuration is the most efficient for storage and retrieval of patterns by the network as a whole. However, in the scale-free case retrieval errors are not distributed uniformly among the nodes. The portion of a pattern encoded by the subset of highly connected nodes is more robust and efficiently recognized than the rest of the pattern. The scale-free network thus achieves a very strong partial recognition. The implications of these findings for brain function and social dynamics are suggestive.
Journal Article↗
Locally activated neural networks and stable neural controller design for nonlinear dynamic systems.
A stable neural control scheme using a locally activated neural network has been proposed for a class of nonlinear dynamic systems. The locally activated neural network, for a given input, essentially selects a small subset of the network hidden nodes for output computation using the CMAC-like content addressing mechanism. This network aims to maintain local representations of the system dynamics. Thus, the global control performance in the concerned state space is achieved by the cooperation of many local control efforts and furthermore, real-time control can be facilitated because only a small sized network is involved to control and learn at any given time. The proposed control scheme is composed of two stages: (1) prediction error based learning in which the network attempts to learn the nonlinear basis functions of the plant inverse dynamics by a modified backpropagation learning rule; and (2) tracking error based learning in which the network weights are further fine-tuned using the basis set obtained in (1). This basis set spans the locally partitioned vector space of the system inverse dynamics when the prediction error based learning is achieved within a prescribed error tolerance. For uniform stability, the sliding mode control is introduced as a safety mechanism when the network has not sufficiently learned the plant dynamics yet. With suitable assumptions on the controlled plant, global stability and tracking error convergence proof has been given. Finally, the proposed control scheme is verified with computer simulation.
Adaptation, Physiological↗
Neural network simulations of coupled locomotor oscillators in the lamprey spinal cord.
The segmental locomotor network in the lamprey spinal cord was simulated on a computer using a connectionist-type neural network. The cells of the network were identical except for their excitatory levels and their synaptic connections. The synaptic connections used were based on previous experimental work. It was demonstrated that the connectivity of the circuit is capable of generating oscillatory activity with the appropriate phase relations among the cells. Intersegmental coordination was explored by coupling two identical segmental networks using only the cells of the network. Each of the possible couplings of a bilateral pair of cells in one oscillator with a bilateral pair of cells in the other oscillator produced stable phase locking of the two oscillators. The degree of phase difference was dependent upon synaptic weight, and the operating range of synaptic weights varied among the pairs of connections. The coupling was tested using several criteria from experimental work on the lamprey spinal cord. Coupling schemes involving several pairs of connecting cells were found which 1) achieved steady-state phase locking within a single cycle, 2) exhibited constant phase differences over a wide range of cycle periods, and 3) maintained stable phase locking in spite of large differences in the intrinsic frequencies of the two oscillators. It is concluded that the synaptic connectivity plays a large role in producing oscillations in this network and that it is not necessary to postulate a separate set of coordinating neurons between oscillators in order to achieve appropriate phase coupling.
Animals↗
Students' use of prior problem representations when performing computer-based simulations: an artificial neural network perspective.
Explore the source record for details and available documents.
Allergy and Immunology↗
Nursing diagnosis by computers: an application of neural networks.
In a pilot-test application of the neural network, test case diagnoses made by the ART-2 neural network agreed with those made independently by nurses. The main conclusion is that the ART-2 neural network is promising as a decision-making aid. It can handle probabilistic and ill-defined data through a process of pattern recognition without programming rules. A disadvantage is that the ART-2 neural network requires "training" by the nurse using at least one example to define a diagnosis. In practice, however, this requirement does not affect its usefulness. Moreover, providing the ART-2 neural network with several training examples enables it to identify new patterns that are "close" to the textbook case but vary in one or more characteristics.
Decision Making, Computer-Assisted↗
Scalar equations for synchronous Boolean networks with biological applications.
One way of coping with the complexity of biological systems is to use the simplest possible models which are able to reproduce at least some nontrivial features of reality. Although two value Boolean models have a long history in technology, it is perhaps a little bit surprising that they can also represent important features of living organizms. In this paper, the scalar equation approach to Boolean network models is further developed and then applied to two interesting biological models. In particular, a linear reduced scalar equation is derived from a more rudimentary nonlinear scalar equation. This simpler, but higher order, two term equation gives immediate information about both cycle and transient structure of the network.
Computational Biology↗
Modular neural networks for non-linearity recovering by the Haar approximation.
The paper deals with the design of a composite neural system for recovering non-linear characteristics from random input-output measurement data. It is assumed that non-linearity output measurements are corrupted by an additive zero-mean white random noise and that the input excitation is an i.i.d. random sequence with an arbitrary (and unknown) probability density function. A class of modular networks is developed. The class is based on the Haar approximation of functions with piecewise constant functions on a refinable grid and consists of the networks composed of perceptron-like modules connected in parallel. The networks provide a local mean value estimators of functions. The relationship between complexity and accuracy of modular networks is analysed. It is shown that under mild conditions on the non-linearities and input probability density functions the networks yield pointwise consistent estimates of non-linear characteristics, provided that complexity of the networks grows appropriately with the number of training data. Efficiency of the networks is examined and the asymptotic rate of convergence of the network estimates is established. Specifically, local ability of the networks to recover non-linear characteristics in dependence on local smoothness of the underlying non-linear function and the input probability density is discussed. Optimum complexity selection rules, guaranteeing the best performance of the networks, are given. Illustrative simulation examples are provided.
Computer Simulation↗
Computer simulation of the segmental neural network generating locomotion in lamprey by using populations of network interneurons.
Realistic computer simulations of the experimentally established local spinal cord neural network generating swimming in the lamprey have been performed. Populations of network interneurons were used in which cellular properties, like cell size and membrane conductance including voltage dependent ion channels were randomly distributed around experimentally obtained mean values, as were synaptic conductances (kainate/AMPA, NMDA, glycine) and delays. This population model displayed more robust burst activity over a wider frequency range than the more simple subsample model used previously, and the pattern of interneuronal activity was appropriate. The strength of the reciprocal inhibition played a very important role in the regulation of burst frequency, and just by changing the inhibitory bias the entire physiological range could be covered. At the lower frequency range of bursting the segmental excitatory interneurons provide stability as does the activation of voltage dependent NMDA receptors. Spike frequency adaptation by means of summation of afterhyperpolarization (AHP) serves as a major burst terminating factor, and at lower rates the membrane properties conferred by the NMDA receptor activation. The lateral interneurons were not of critical importance for the burst termination. They may, however, be of particular importance for inducing a rapid burst termination during for instance steering and righting reactions. Several cellular factors combine to provide a secure and stable motor pattern in the entire frequency range.
Animals↗
SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons.
Many biological neural network models face the problem of scalability because of the limited computational power of today's computers. Thus, it is difficult to assess the efficiency of these models to solve complex problems such as image processing. Here, we describe how this problem can be tackled using event-driven computation. Only the neurons that emit a discharge are processed and, as long as the average spike discharge rate is low, millions of neurons and billions of connections can be modelled. We describe the underlying computation and implementation of such a mechanism in SpikeNET, our neural network simulation package. The type of model one can build is not only biologically compliant, it is also computationally efficient as 400 000 synaptic weights can be propagated per second on a standard desktop computer. In addition, for large networks, we can set very small time steps (< 0.01 ms) without significantly increasing the computation time. As an example, this method is applied to solve complex cognitive tasks such as face recognition in natural images.
Action Potentials↗
Neural network pruning with Tukey-Kramer multiple comparison procedure.
Reducing a neural network's complexity improves the ability of the network to generalize future examples. Like an overfitted regression function, neural networks may miss their target because of the excessive degrees of freedom stored up in unnecessary parameters. Over the past decade, the subject of pruning networks produced nonstatistical algorithms like Skeletonization, Optimal Brain Damage, and Optimal Brain Surgeon as methods to remove connections with the least salience. The method proposed here uses the bootstrap algorithm to estimate the distribution of the model parameter saliences. Statistical multiple comparison procedures are then used to make pruning decisions. We show this method compares well with Optimal Brain Surgeon in terms of ability to prune and the resulting network performance.
Algorithms↗
Interpolation and extrapolation in human behavior and neural networks.
Unlike most artificial systems, the brain is able to face situations that it has not learned or even encountered before. This ability is not in general echoed by the properties of most neural networks. Here, we show that neural computation based on least-square error learning between populations of intensity-coded neurons can explain interpolation and extrapolation capacities of the nervous system in sensorimotor and cognitive tasks. We present simulations for function learning experiments, auditory-visual behavior, and visuomotor transformations. The results suggest that induction in human behavior, be it sensorimotor or cognitive, could arise from a common neural associative mechanism.
Adaptation, Psychological↗
Modeling neuronal assemblies: theory and implementation.
Models that describe qualitatively and quantitatively the activity of entire groups of spiking neurons are becoming increasingly important for biologically realistic large-scale network simulations. At the systems and areas modeling level, it is necessary to switch the basic descriptional level from single spiking neurons to neuronal assemblies. In this article, we present and review work that allows a macroscopic description of the assembly activity. We show that such macroscopic models can be used to reproduce in a quantitatively exact manner the joint activity of groups of spike-response or integrate-and-fire neurons. We also show that integral as well as differential equation models of neuronal assemblies can be understood within a single framework, which allows a comparison with the commonly used assembly-averaged graded-response type of models. The presented framework thus enables the large-scale neural network modeler to implement networks using computational units beyond the single spiking neuron without losing much biological accuracy. This article explains the theoretical background as well as the capabilities and the implementation details of the assembly approach.
Computer Simulation↗
Pulse stream VLSI circuits and systems: the EPSILON neural network chipset.
An analogue CMOS VLSI neural processing chip has been designed and fabricated. The device employs "pulse-stream" neural state signalling and is capable of computing some 360 million synaptic connections per second. In addition to basic characterisation results, the performance of the chip in solving "real-world" problems is also demonstrated. The experience gained from the development of this device has resulted in the design of a second "pulse-stream" chip with improved performance and features. It is anticipated that this second device will be integrated into a standard bus-based system and find early application in robotic control.
Computers, Analog↗
Instabilities and oscillation in the deterministic Boltzmann machine.
Simulations indicate that the deterministic Boltzmann machine, unlike the stochastic Boltzmann machine from which it is derived, exhibits unstable behavior during contrastive Hebbian learning of nonlinear problems, including oscillation in the learning algorithm and extreme sensitivity to small weight perturbations. Although careful choice of the initial weight magnitudes, the learning rate, and the annealing schedule will produce convergence in most cases, the stability of the resulting solution depends on the parameters in a complex and generally indiscernible way. We show that this unstable behavior is the result of over parameterization (excessive freedom in the weights), which leads to continuous rather than isolated optimal weight solution sets. This allows the weights to drift without correction by the learning algorithm until the free energy landscape changes in such a way that the settling procedure employed finds a different minimum of the free energy function than it did previously and a gross output error occurs. Because all the weight sets in a continuous optimal solution set produce exactly the same network outputs, we define reliability, a measure of the robustness of the network, as a new performance criterion.
Algorithms↗
XNBC: a simulation tool. Application to the study of neural coding using hybrid networks.
XNBC is a software package for simulating biological neural networks. Two neuron models are available, a leaky integrator model and an ion-conductance model. Inputs to the simulated neurons can be provided by experimental data stored in files, allowing the creation of 'hybrid' networks. Graphic tools are used to describe the modeled neurons as well as the network. Neuron and network parameters can be modified during the simulation, to mimic electrical stimulations and drugs action. The temporal evolution of the network and of selected neurons can be visualized. A point process, frequency or dynamic analysis of the simulator output can be performed. The successive stages of the creation of a hybrid network are explained.
Computer Graphics↗
Transport in weighted networks: partition into superhighways and roads.
Transport in weighted networks is dominated by the minimum spanning tree (MST), the tree connecting all nodes with the minimum total weight. We find that the MST can be partitioned into two distinct components, having significantly different transport properties, characterized by centrality--the number of times a node (or link) is used by transport paths. One component, superhighways, is the infinite incipient percolation cluster, for which we find that nodes (or links) with high centrality dominate. For the other component, roads, which includes the remaining nodes, low centrality nodes dominate. We find also that the distribution of the centrality for the infinite incipient percolation cluster satisfies a power law, with an exponent smaller than that for the entire MST. The significance of this finding is that one can improve significantly the global transport by improving a tiny fraction of the network, the superhighways.
Computer Simulation↗
Using a feed-forward network to incorporate the relation between attractees and attractors in a generalized discrete Hopfield network.
This paper demonstrates how a feedforward network with constant connection matrices may be used to train a Hopfield style network for pattern recognition. The connection matrix of the Hopfield style network is asymmetric and its diagonal is non-zero. The Hopfield style network referred to as a GDHN is trained to incorporate a relation between attractees and attractors. The attractees represent class samples and the attractors represent class prototypes. The feedforward network is trained using a gradient descent method. Gradients are fed forward in the network to obtain a gradient for a cost function.
Algorithms↗