Biomedical subjects

D J Amit

Publications and source records attributed to D J Amit.

10 recordsLinked to original sources

PubMed2000

Spike-driven synaptic plasticity: theory, simulation, VLSI implementation.

We present a model for spike-driven dynamics of a plastic synapse, suited for aVLSI implementation. The synaptic device behaves as a capacitor on short timescales and preserves the memory of two stable states (efficacies) on long timescales. The transitions (LTP/LTD) are stochastic because both the number and the distribution of neural spikes in any finite (stimulation) interval fluctuate, even at fixed pre- and postsynaptic spike rates. The dynamics of the single synapse is studied analytically by extending the solution to a classic problem in queuing theory (Takacs process). The model of the synapse is implemented in aVLSI and consists of only 18 transistors. It is also directly simulated. The simulations indicate that LTP/LTD probabilities versus rates are robust to fluctuations of the electronic parameters in a wide range of rates. The solutions for these probabilities are in very good agreement with both the simulations and measurements. Moreover, the probabilities are readily manipulable by variations of the chip's parameters, even in ranges where they are very small. The tests of the electronic device cover the range from spontaneous activity (3-4 Hz) to stimulus-driven rates (50 Hz). Low transition probabilities can be maintained in all ranges, even though the intrinsic time constants of the device are short (approximately 100 ms). Synaptic transitions are triggered by elevated presynaptic rates: for low presynaptic rates, there are essentially no transitions. The synaptic device can preserve its memory for years in the absence of stimulation. Stochasticity of learning is a result of the variability of interspike intervals; noise is a feature of the distributed dynamics of the network. The fact that the synapse is binary on long timescales solves the stability problem of synaptic efficacies in the absence of stimulation. Yet stochastic learning theory ensures that it does not affect the collective behavior of the network, if the transition probabilities are low and LTP is balanced against LTD.

Action Potentials↗

PubMed1999

What is and what is not a theory of context correlations.

Recently a feed-forward model was put forward to account for the phenomenon of context correlations (observed by Miyashita and by Yakovlev et al) to replace the attractor picture developed for these findings. It is argued that the new proposal misses the main issues posed by the experimental situation as well as the salient aspects of the attractor scenario, which combines delay activity neural dynamics with learning at-a-distance.

Action Potentials↗

PubMed1999

Effective neural response function for collective population states.

Collective behaviour of neural networks often divides the ensemble of neurons into sub-classes by neuron type; by selective synaptic potentiation; or by mode of stimulation. When the number of classes becomes larger than two, the analysis, even in a mean-field theory, loses its intuitive aspect because of the number of dimensions of the space of dynamical variables. Often one is interested in the behaviour of a reduced set of sub-populations (in focus) and in their dependence on the system's parameters, as in searching for coexistence of spontaneous activity and working memory; in the competition between different working memories; in the competition between working memory and a new stimulus; or in the interaction between selective activity in two different neural modules. For such cases we present a method for reducing the dimensionality of the system to one or two dimensions, even when the total number of populations involved is higher. In the reduced system the familiar intuitive tools apply and the analysis of the dependence of different network states on ambient parameters becomes transparent. Moreover, when the coding of states in focus is sparse, the computational complexity is much reduced. Beyond the analysis, we present a set of detailed examples. We conclude with a discussion of questions of stability in the reduced system.

Animals↗

PubMed1998

Simulation in neurobiology: theory or experiment?

Investigation in neurophysiology usually involves measurements of large population-average signals or small sample recordings. There is an underlying assumption that the observations express activity of large groups of similarly acting neurons that is the result of a bottom-up scenario in which individual cells, via their synaptic interactions, lead to the large scale phenomena. The connection between the levels must be provided by theory, which must also provide the relevant variables for observation. It is suggested that between the experiment and the full theory there is a creative, mixed role for simulation: both experimental and theoretical. A simulation presents complex dynamics and hence is an empirical board for testing theoretical tools, yet its controlled behaviour can make predictions about the biological system.

Animals↗

PubMed1998

Learning attractors in an asynchronous, stochastic electronic neural network.

LANN27 is an electronic device implementing in discrete electronics a fully connected (full feedback) network of 27 neurons and 351 plastic synapses with stochastic Hebbian learning. Both neurons and synapses are dynamic elements, with two time constants--fast for neurons and slow for synapses. Learning, synaptic dynamics, is analogue and is driven in a Hebbian way by neural activities. Long-term memorization takes place on a discrete set of synaptic efficacies and is effected in a stochastic manner. The intense feedback between the nonlinear neural elements, via the learned synaptic structure, creates in an organic way a set of attractors for the collective retrieval dynamics of the neural system, akin to Hebbian learned reverberations. The resulting structure of the attractors is a record of the large-scale statistics in the uncontrolled, incoming flow of stimuli. As the statistics in the stimulus flow changes significantly, the attractors slowly follow it and the network behaves as a palimpsest--old is gradually replaced by new. Moreover, the slow learning creates attractors which render the network a prototype extractor: entire clouds of stimuli, noisy versions of a prototype, used in training, all retrieve the attractor corresponding to the prototype upon retrieval. Here we describe the process of studying the collective dynamics of the network, before, during and following learning, which is rendered complex by the richness of the possible stimulus streams and the large dimensionality of the space of states of the network. We propose sampling techniques and modes of representation for the outcome.

Animals↗

PubMed1997

Paradigmatic working memory (attractor) cell in IT cortex.

We discuss paradigmatic properties of the activity of single cells comprising an attractor-a developed stable delay activity distribution. To demonstrate these properties and a methodology for measuring their values, we present a detailed account of the spike activity recorded from a single cell in the inferotemporal cortex of a monkey performing a delayed match-to-sample (DMS) task of visual images. In particular, we discuss and exemplify (1) the relation between spontaneous activity and activity immediately preceding the first stimulus in each trial during a series of DMS trials, (2) the effect on the visual response (i.e., activity during stimulation) of stimulus degradation (moving in the space of IT afferents), (3) the behavior of the delay activity (i.e., activity following visual stimulation) under stimulus degradation (attractor dynamics and the basin of attraction), and (4) the propagation of information between trials-the vehicle for the formation of (contextual) correlations by learning a fixed stimulus sequence (Miyashita, 1988). In the process of the discussion and demonstration, we expose effective tools for the identification and characterization of attractor dynamics.

Animals↗

PubMed1994

Correlations of cortical Hebbian reverberations: theory versus experiment.

Interpreting recent single-unit recordings of delay activities in delayed match-to-sample experiments in anterior ventral temporal (AVT) cortex of monkeys in terms of reverberation dynamics, we present a model neural network of quasi-realistic elements that reproduces the empirical results in great detail. Information about the contiguity of successive stimuli in the training sequence, representing the fact that training is done on a set of uncorrelated stimuli presented in a fixed temporal sequence, is embedded in the synaptic structure. The model reproduces quite accurately the correlations between delay activity distributions corresponding to stimulation with the uncorrelated stimuli used for training. It reproduces also the activity distributions of spike rates on sample cells as a function of the stimulating pattern. It is, in our view, the first time that a computational phenomenon, represented on the neurophysiological level, is reproduced in all its quantitative aspects. The model is then used to make predictions about further features of the physiology of such experiments. Those include further properties of the correlations, features of selective cells as discriminators of stimuli provoking different delay activity distributions, and activity distributions among the neurons in a delay activity produced by a given pattern. The model has predictive implications also for the dependence of the delay activities on different training protocols. Finally, we discuss the perspectives of the interplay between such models and neurophysiology as well as its limitations and possible extensions.

Action Potentials↗

PubMed1989

Associative memory neural network with low temporal spiking rates.

We describe a modified attractor neural network in which neuronal dynamics takes place on a time scale of the absolute refractory period but the mean temporal firing rate of any neuron in the network is lower by an arbitrary factor that characterizes the strength of the effective inhibition. It operates by encoding information on the excitatory neurons only and assuming the inhibitory neurons to be faster and to inhibit the excitatory ones by an effective postsynaptic potential that is expressed in terms of the activity of the excitatory neurons themselves. Retrieval is identified as a nonergodic behavior of the network whose consecutive states have a significantly enhanced activity rate for the neurons that should be active in a stored pattern and a reduced activity rate for the neurons that are inactive in the memorized pattern. In contrast to the Hopfield model the network operates away from fixed points and under the strong influence of noise. As a consequence, of the neurons that should be active in a pattern, only a small fraction is active in any given time cycle and those are randomly distributed, leading to reduced temporal rates. We argue that this model brings neural network models much closer to biological reality. We present the results of detailed analysis of the model as well as simulations.

Animals↗

PubMed1988

Neural networks counting chimes.

It is shown that the ideas that led to neural networks capable of recalling associatively and asynchronously temporal sequences of patterns can be extended to produce a neural network that automatically counts the cardinal number in a sequence of identical external stimuli. The network is explicitly constructed, analyzed, and simulated. Such a network may account for the cognitive effect of the automatic counting of chimes to tell the hour. A more general implication is that different electrophysiological responses to identical stimuli, at certain stages of cortical processing, do not necessarily imply synaptic modification, a la Hebb. Such differences may arise from the fact that consecutive identical inputs find the network in different stages of an active temporal sequence of cognitive states. These types of networks are then situated within a program for the study of cognition, which assigns the detection of meaning as the primary role of attractor neural networks rather than computation, in contrast to the parallel distributed processing attitude to the connectionist project. This interpretation is free of homunculus, as well as from the criticism raised against the cognitive model of symbol manipulation. Computation is then identified as the syntax of temporal sequences of quasi-attractors.

Computer Simulation↗

PubMedDate not supplied

Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex.

We investigate self-sustaining stable states (attractors) in networks of integrate-and-fire neurons. First, we study the stability of spontaneous activity in an unstructured network. It is shown that the stochastic background activity, of 1-5 spikes/s, is unstable if all neurons are excitatory. On the other hand, spontaneous activity becomes self-stabilizing in presence of local inhibition, given reasonable values of the parameters of the network. Second, in a network sustaining physiological spontaneous rates, we study the effect of learning in a local module, expressed in synaptic modifications in specific populations of synapses. We find that if the average synaptic potentiation (LTP) is too low, no stimulus specific activity manifests itself in the delay period. Instead, following the presentation and removal of any stimulus there is, in the local module, a delay activity in which all neurons selective (responding visually) to any of the stimuli presented for learning have rates which gradually increase with the amplitude of synaptic potentiation. When the average LTP increases beyond a critical value, specific local attractors (stable states) appear abruptly against the background of the global uniform spontaneous attractor. In this case the local module has two available types of collective delay activity: if the stimulus is unfamiliar, the activity is spontaneous; if it is similar to a learned stimulus, delay activity is selective. These new attractors reflect the synaptic structure developed during learning. In each of them a small population of neurons have elevated rates, which depend on the strength of LTP. The remaining neurons of the module have their activity at spontaneous rates. The predictions made in this paper could be checked by single unit recordings in delayed response experiments.

Cerebral Cortex↗