Some developmental and attentional biases in the contrast enhancement and short term memory of recurrent neural networks.
Explore the source record for details and available documents.
Memory, Short-Term↗
SEARCH · PubMed Health
Results for “recurrent neural network”
Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Towards mechanistic models of mutational effects: Deep learning on Alzheimer's Aβ peptide.
Deep Mutational Scanning (DMS) has enabled multiplexed measurement of mutational effects on protein properties, including kinematics and self-organization, with unprecedented resolution. However, potential bottlenecks of DMS characterization include experimental design, data quality, and depth of mutational coverage. Here, we apply deep learning to comprehensively model the mutational effect of the Alzheimer's Disease associated peptide Aβ42 on aggregation-related biochemical traits from DMS measurements. Among tested neural network architectures, Convolutional Neural Networks and Recurrent Neural Networks are found to be the most cost-effective models with high performance even under insufficiently-sampled DMS studies. While sequence features are essential for satisfactory prediction from neural networks, geometric-structural features further enhance the prediction performance. Notably, we demonstrate how mechanistic insights into phenotype may be extracted from the neural networks themselves suitably designed. This methodological benefit is particularly relevant for biochemical systems displaying a strong coupling between structure and phenotype such as the conformation of Aβ42 aggregate and nucleation, as shown here using a Graph Convolutional Neural Network (GCN) developed from the protein atomic structure input. In addition to accurate imputation of missing values (which here ranged up to 55% of all phenotype values at key residues), the mutationally-defined nucleation phenotype generated from a GCN shows improved resolution for identifying known disease-causing mutations relative to the original DMS phenotype. Our study suggests that neural network derived sequence-phenotype mapping can be exploited not only to provide direct support for protein engineering or genome editing but also to facilitate therapeutic design with the gained perspectives from biological modeling.
Alzheimer's disease↗
Universality in neural networks: the importance of the 'mean firing rate'.
We present a general analysis of highly connected recurrent neural networks which are able to learn and retrieve a finite number of static patterns. The arguments are based on spike trains and their interval distribution and require no specific model of a neuron. In particular, they apply to formal two-state neurons as well as to more refined models like the integrate-and-fire neuron or the Hodgkin-Huxley equations. We show that the mean firing rate defined as the inverse of the mean interval length is the only relevant parameter (apart from the synaptic weights) that determines the existence of retrieval solutions with a large overlap with one of the learnt patterns. The statistics of the spiking noise (Gaussian, Poisson or other) and hence the shape of the interval distribution does not matter. Thus our unifying approach explains why, and when, all the different associative networks which treat static patterns yield basically the same results, i.e., belong to the same universality class.
Mathematics↗
Motor learning in a recurrent network model based on the vestibulo-ocular reflex.
Most models of neural networks have assumed that neurons process information on a timescale of milliseconds and that the long-term modification of synaptic strengths underlies learning and memory. But neurons also have cellular mechanisms that operate on a timescale of tens or hundreds of milliseconds, such as a gradual rise in firing rate in response to injection of constant current or a rapid rise followed by a slower adaptation. These dynamic properties of neuronal responses are mediated by ion channels that are subject to modulation. We demonstrate here how a neural network with recurrent feedback connections can convert long-term modulation of neural responses that occur over these intermediate timescales into changes in the amplitude of the steady output from the system. This general principle may be relevant to many feedback systems in the brain. Here it is applied to the vestibulo-ocular reflex, whose amplitude is subject to long-term adaptive modification by visual inputs. The model reconciles apparently contradictory data on the neural locus of the cellular mechanisms that mediate this simple form of learning and memory.
Animals↗
A demonstration that breast cancer recurrence can be predicted by neural network analysis.
Neural Network Analysis, a form of artificial intelligence, was successfully used to predict the clinical outcome of node-positive breast cancer patients. A Neural Network was trained to predict clinical outcome using prognostic information from 1008 patients. During training, the network received as input information tumor hormone receptor status, DNA index and S-phase determination by flow cytometry, tumor size, number of axillary lymph nodes involved with tumor, and age of the patient, as well as length of clinical followup, relapse status, and time of relapse. The ability of the trained Network to determine relapse probability was then validated in a separate set of 960 patients. The Neural Network was as powerful as Cox Regression Modeling in identifying breast cancer patients at high and low risk for relapse.
Axilla↗
Learning neural dynamics through instructive signals.
Rapid learning is essential for flexible behavior, but its basis in the brain remains unknown. Here we introduce the PRISM plasticity rule, a unifying mechanistic model of three well-established, fast-acting synaptic plasticity rules-in hippocampus, cerebellum and mushroom body-which relies exclusively on pre-synaptic activity and an "instructive signal" from another brain area. Using a multi-region network model we show that guiding PRISM plasticity with instructive signals enables the network to quickly learn extremely flexible nonlinear dynamics underlying behaviorally relevant computations, as well as to emulate unknown external system dynamics from real-time error signals, which we demonstrate with comprehensive simulations supported by exact mathematical theory. Thus, PRISM plasticity guided by instructive signals is well-suited to rapidly learn general-purpose neural computations-in contrast to canonical Hebbian rules. Finally, we show how including this plasticity rule in artificial learning algorithms can solve long-range temporal credit assignment, a long-standing challenge in machine learning.
cerebellum↗
Associative recall and formation of stable modes of activity in neural network models.
Models of neural networks with recurrent inhibition are studied, as well as one model which also includes recurrent excitation. The models are intended as possible descriptions of the cerebral cortex. Each network model is composed of neuron models called pyramidal cells and stellate cells in accordance with the names of two types of cells in the cortex. Inputs and outputs of the network are connected to the pyramidal cells while feedback is provided by the stellate cells. Connections within the network are random. During a learning phase the pyramidal cell excitatory synapses become facilitated according to a two-conditional facilitation rule. This is the basis of the model's ability for associative learning. The associative retrieval of information can be studied during a subsequent association phase. This has been done by simulation on a digital computer. It was shown that all of the models considered can be designed to perform a so-called decision-making function. This means that if the associating input pattern is similar to several patterns which occurred during learning the model can decide which similarity is greatest by responding with the appropriate associated pattern. The model also including recurrent excitation differs from the simpler models in that it can become stabilized in so-called stable modes of activity which are self-sustaining and remain even after the input has been turned off. Normally, only one stable mode can be active at a time. However, through careful choice of construction parameters it was possible to obtain a model in which a maximum of two stable modes could be activated independently of each other. Physiological and psychological interpretations are discussed and so are the limitations of the models, which are evident in certain situations.
Cerebral Cortex↗
Treatment decisions in axillary node-negative breast cancer patients.
Treatment decisions must be made on 9000 axillary node-negative breast cancer patients each month in the United States. Which of these patients will benefit from adjuvant therapy is a major question. Valid methods are needed to distinguish those patients who are "cured" from those who will suffer a cancer recurrence. A complex network of prognostic variables enters into the treatment decision, together with a risk-versus-benefit assessment. We are using a neural-network-based form of artificial intelligence that, once "trained" with data representing an event and its outcome, can identify subsets of patients with low recurrence risks. Larger data sets are being evaluated with the hope of introducing the neural-network technique to routine clinical practice.
Breast Neoplasms↗
A practical application of neural network analysis for predicting outcome of individual breast cancer patients.
It has been previously shown that Neural Networks can be trained to recognize individual breast cancer patients at high and low risk for recurrent disease and death. This paper expands on the initial investigation and shows that by coding time as one of the prognostic variables, a Neural Network can use censored survival data to predict patient outcome over time. In this demonstration a Neural Network was trained, tested, and validated using censored survival data from a group of 1373 patients with node-positive breast cancer. The Neural Network method predicted patient outcome as accurately as Cox Regression modeling. The final Neural Network model can be presented with a patient's prognostic information and make a series of predictions about probability of relapse at different times of follow-up, allowing one to draw survival probability curves for individual patients.
Adult↗
Simulating vestibular compensation using recurrent back-propagation.
Vestibular compensation is simulated as learning in a dynamic neural network model of the horizontal vestibulo-ocular reflex (VOR). The bilateral, three-layered VOR model consists of nonlinear units representing horizontal canal afferents, vestibular nuclei (VN) neurons and eye muscle motoneurons. Dynamic processing takes place via commissural connections that link the VN bilaterally. The intact network is trained, using recurrent back-propagation, to produce the VOR with velocity storage integration. Compensation is simulated by removing vestibular afferent input from one side and retraining the network. The time course of simulated compensation matches that observed experimentally. The behavior of model VN neurons in the compensated network also matches real data, but only if connections at the motoneurons, as well as at the VN, are allowed to be plastic. The dynamic properties of real VN neurons in compensated and normal animals are found to differ when tested with sinusoidal but not with step stimuli. The model reproduces these conflicting data, and suggests that the disagreement may be due to VN neuron nonlinearity.
Algorithms↗
Electrosensory systems in fish.
A close integration of behavioral, neurophysiological, and neuroanatomical approaches has guided research on the neural basis of electrosensation and the generation of behaviors associated with this modality. By postulating neuronal implementations of specific computations in sensory information processing, behavioral studies have been crucial in focusing studies at the neuronal level onto behaviorally relevant structural and functional aspects. Physiological and anatomical studies have analyzed a) neural networks underlying the distributed processing of sensory information, b) the role of descending recurrent pathways and efference copy mechanisms for the filtering of incoming information, c) the significance of multiple topographic representations for sensory information processing, and d) the modulation of sensory and motor structures through various transmitters and receptor subtypes. Developmental studies have explored the significance of steroid hormones for the tuning of electroreceptors to the frequency of an endogenous neuronal oscillator which drives the electric current pulses necessary for their stimulation. Embryological studies have revealed that the development of mechanoreceptors and electroreceptors in the fish's skin is induced by the innervation of primary afferent nerve fibers which are specific with regard to their central connections as well as with regard to the type of receptor induced in the periphery.
Animals↗
Inputs from the posterior lateral line nerves upon the goldfish Mauthner cells. II. Evidence that the inhibitory components are mediated by interneurons of the recurrent collateral network.
Explore the source record for details and available documents.
Animals↗
Neural networks and synaptic transmission in immature hippocampus.
The results reviewed in this chapter indicate that local circuit synaptic interactions are surprisingly well-developed in the rat hippocampal CA3 subfield during the second postnatal week. Intracellular recordings reveal large spontaneous epsps and ipsps. Synchronized bursts of synaptic potentials are observed in most paired intracellular recordings. Antidromic and orthodromic electrical stimulation evokes synaptic responses that are reminiscent of recordings from mature hippocampus. However, following brief trains of electrical stimuli and during bath application of a GABAA receptor antagonist, large prolonged depolarizations are recorded. These are suppressed by excitatory amino acid antagonists. In comparison, slices from mature rats do not produce these events under the same conditions. Thus, a hypothesis has been presented that the degree of recurrent synaptic interaction between pyramidal cells may be enhanced during this critical period in hippocampal development. An analysis of recurrent epsps using dual intracellular recordings is consistent with this contention. The degree of excitatory synaptic interaction between CA3 neurons appears to be at least equal to and likely in excess of that reported in mature hippocampus. One possible explanation for this is that the number of recurrent excitatory synapses may increase transiently during hippocampal development, only later to regress to numbers found in the adult. Recent studies of others suggest that activation of NMDA receptors may play a key role in the maintenance of synapses during development (Cline et al., 1987; Kleinschmidt et al., 1987). In this regard it is interesting that the characteristic of the NMDA receptor-ion channel complex on immature and mature rat CA3 hippocampal pyramidal cells appear to be quite different. These differences may play a role in synapse formation and maintenance. The role of NMDA receptors in LTP and learning (for review see Cotman et al., 1989; Lynch, 1986) are widely discussed. In other circles recurrent excitatory neuronal networks are thought to be substrates for memory (Lynch, 1986; Haberly and Bower, 1989). Both NMDA receptors and recurrent excitation are well represented in the CA3 subfield of immature hippocampus. One challenging area for future study will be the clarification of the interrelations between synaptogenesis and synaptic plasticity within neural networks of the developing hippocampus.
Animals↗
Instability in a hippocampal neural network.
Four computer models, HM1-HM4, of a particular hippocampal neural network have been developed. The models represent two cell populations, the pyramidal cells and the basket cells; the populations are coupled so that pyramidal cells are inhibited by activity they excite in the basket cell population. In models HM2-HM4, this recurrent inhibitory pathway contains a temporal dispersion element. Models HM3-HM4 place the pyramidal cells in an additional recurrent excitatory feedback loop. HM4 represents a pair of interacting hippocampal networks. Simulated network responses to single-shock stimulation are presented for various parameter values of the four models. Calculations are extended more than one second following stimulation. Particular attention is given to simulation of network instabilities. Simulated neural activity is discussed in view of experimental work on normal (nonepileptogenic) and epileptogenic hippocampal cortex.
Computers↗
[The long-term action of camphor on "audiosensitive" rats: electrophysiological research and mathematical modelling of the properties of the neuronal networks].
Rats of the Krushinskiĭ-Molodkina (KM) strain genetically predisposed to audiogenic seizures were repeatedly injected camphor for 4-5 months in gradually increasing doses (beginning from those below the seizure threshold up to those above it). As a result of "adaptation" to the convulsant action rats endured without seizures camphor doses 1.5-3 times higher than seizure threshold ones. Parallel with this "adaptation" the motor cortex of KM rats accumulated properties peculiar to the epileptic focus: spontaneous epileptiform spikes were recorded there after the camphor application. Estimation of the neural network parameters from spectra of the motor cortex biopotentials using the network model has shown a decrease of a parameter reflecting efficacy of the recurrent inhibition mechanisms. It is supposed that development of compensatory processes making rats able to avoid generalized camphor seizures are related to an increase of inhibitory functions of the caudate nucleus and weakening of the hippocampal excitability.
Acoustic Stimulation↗
Complex dynamics and noise in simple neural networks with delayed mixed feedback.
This paper briefly reviews the role of mixed feedback, neural delays, and neural noise in the genesis of complex oscillations in neurological feedback systems. The results are concretely discussed within the context of recurrent inhibition in the mammalian hippocampus, and a hybrid version of the pupil light reflex with externally imposed electronic feedback.
Animals↗
Self-organization of associative memory and pattern classification: recurrent signal processing on topological feature maps.
We extend the neural concepts of topological feature maps towards self-organization of auto-associative memory and hierarchical pattern classification. As is well-known, topological maps for statistical data sets store information on the associated probability densities. To extract that information we introduce a recurrent dynamics of signal processing. We show that the dynamics converts a topological map into an auto-associative memory for real-valued feature vectors which is capable to perform a cluster analysis. The neural network scheme thus developed represents a generalization of non-linear matrix-type associative memories. The results naturally lead to the concept of a feature atlas and an associated scheme of self-organized, hierarchical pattern classification.
Algorithms↗
Spatial propagation of associations in a cortex-like neural network model.
A neural network model is studied, having associative memory properties and allowing retrieved associations to propagate within the network. It is intended as a tentative description of the cerebral cortex consisting of "pyramidal cells" with modifiable synapses and "stellate cells" providing feedback through excitatory and inhibitory recurrent pathways. The model is based on some general assumptions: Learning occurs through facilitation of synapses which depends on simultaneous pre- and postsynaptic activity (two-conditional facilitation). Connections within the network are realizations of a random process, implying that nearby cells are more likely to be connected than distant one. The two-conditional facilitation makes it possible for an output signal pattern which occurred in conjunction with a certain input pattern to be retrieved later by reapplying the particular input, the model working as an associative memory. The random connections and the operation of the stellate cell models as linear threshold units give rise to pattern separation in the feedback link. This, in addition to the fact that patterns form associations with themselves, is of importance during the associative recall enabling the network to attain alternative stable modes of activity each corresponding to a learned association. It is shown that a learned pattern of activity which is retrieved, ie, a stable mode, can propagate across the surface of the network. The mode of activity evoked through a certain association may get into contact with modes originating from different associations, forming a stable or slowly moving boundary between the interacting modes. The model is discussed in relation to some properties of the visual system.
Association↗