Biomedical subjects

H Sebastian Seung

Publications and source records attributed to H Sebastian Seung.

At least 19 recordsLinked to original sources

PubMed2006

Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity.

The probability of choosing an alternative in a long sequence of repeated choices is proportional to the total reward derived from that alternative, a phenomenon known as Herrnstein's matching law. This behavior is remarkably conserved across species and experimental conditions, but its underlying neural mechanisms still are unknown. Here, we propose a neural explanation of this empirical law of behavior. We hypothesize that there are forms of synaptic plasticity driven by the covariance between reward and neural activity and prove mathematically that matching is a generic outcome of such plasticity. Two hypothetical types of synaptic plasticity, embedded in decision-making neural network models, are shown to yield matching behavior in numerical simulations, in accord with our general theorem. We show how this class of models can be tested experimentally by making reward not only contingent on the choices of the subject but also directly contingent on fluctuations in neural activity. Maximization is shown to be a generic outcome of synaptic plasticity driven by the sum of the covariances between reward and all past neural activities.

Animals↗

PubMed2006

Neural voting machines.

A "Winner-take-all" network is a computational mechanism for picking an alternative with the largest excitatory input. This choice is far from optimal when there is uncertainty in the strength of the inputs, and when information is available about how alternatives may be related. For some time, the Social Choice community has recognized that many other procedures will yield more robust winners. The Borda Count and the pair-wise Condorcet tally are among the most favored. If biological systems strive to optimize information aggregation, then it is of interest to examine the complexity of networks that implement these procedures. We offer two biologically feasible implementations that are relatively simple modifications of classical recurrent networks.

Choice Behavior↗

PubMed2006

Gradient learning in spiking neural networks by dynamic perturbation of conductances.

We present a method of estimating the gradient of an objective function with respect to the synaptic weights of a spiking neural network. The method works by measuring the fluctuations in the objective function in response to dynamic perturbation of the membrane conductances of the neurons. It is compatible with recurrent networks of conductance-based model neurons with dynamic synapses. The method can be interpreted as a biologically plausible synaptic learning rule, if the dynamic perturbations are generated by a special class of "empiric" synapses driven by random spike trains from an external source.

Action Potentials↗

PubMed2005

Tilt aftereffect and adaptation-induced changes in orientation tuning in visual cortex.

The tilt aftereffect (TAE) is a visual illusion in which prolonged adaptation to an oriented stimulus causes shifts in subsequent perceived orientations. Historically, neural models of the TAE have explained it as the outcome of response suppression of neurons tuned to the adapting orientation. Recent physiological studies of neurons in primary visual cortex (V1) have confirmed that such response suppression exists. However, it was also found that the preferred orientations of neurons shift away from the adapting orientation. Here we show that adding this second factor to a population coding model of V1 improves the correspondence between neurophysiological data and TAE measurements. According to our model, the shifts in preferred orientation have the opposite effect as response suppression, reducing the magnitude of the TAE.

Adaptation, Physiological↗

PubMed2005

Learning curves for stochastic gradient descent in linear feedforward networks.

Gradient-following learning methods can encounter problems of implementation in many applications, and stochastic variants are sometimes used to overcome these difficulties. We analyze three online training methods used with a linear perceptron: direct gradient descent, node perturbation, and weight perturbation. Learning speed is defined as the rate of exponential decay in the learning curves. When the scalar parameter that controls the size of weight updates is chosen to maximize learning speed, node perturbation is slower than direct gradient descent by a factor equal to the number of output units; weight perturbation is slower still by an additional factor equal to the number of input units. Parallel perturbation allows faster learning than sequential perturbation, by a factor that does not depend on network size. We also characterize how uncertainty in quantities used in the stochastic updates affects the learning curves. This study suggests that in practice, weight perturbation may be slow for large networks, and node perturbation can have performance comparable to that of direct gradient descent when there are few output units. However, these statements depend on the specifics of the learning problem, such as the input distribution and the target function, and are not universally applicable.

Neural Networks, Computer↗

PubMed2005

Linear-Nonlinear-Poisson models of primate choice dynamics.

The equilibrium phenomenon of matching behavior traditionally has been studied in stationary environments. Here we attempt to uncover the local mechanism of choice that gives rise to matching by studying behavior in a highly dynamic foraging environment. In our experiments, 2 rhesus monkeys (Macacca mulatta) foraged for juice rewards by making eye movements to one of two colored icons presented on a computer monitor, each rewarded on dynamic variable-interval schedules. Using a generalization of Wiener kernel analysis, we recover a compact mechanistic description of the impact of past reward on future choice in the form of a Linear-Nonlinear-Poisson model. We validate this model through rigorous predictive and generative testing. Compared to our earlier work with this same data set, this model proves to be a better description of choice behavior and is more tightly correlated with putative neural value signals. Refinements over previous models include hyperbolic (as opposed to exponential) temporal discounting of past rewards, and differential (as opposed to fractional) comparisons of option value. Through numerical simulation we find that within this class of strategies, the model parameters employed by animals are very close to those that maximize reward harvesting efficiency.

Animals↗

PubMed2004

Light-directed electrical stimulation of neurons cultured on silicon wafers.

Dissociated neurons cultured in vitro can serve as a model system for studying the dynamics of neural networks. Such studies depend on techniques for stimulating patterns of neural activity. We show a technique for extracellular stimulation of dissociated neurons cultured on silicon wafers. When the silicon surface is reverse biased, electrical current can be generated near any neuron by pulsing a laser. Complex spatiotemporal stimulation patterns can be produced by directing a single beam with an acousto-optic deflector. The technique can generate a stimulating current at any location in the culture. This contrasts with multielectrode arrays (MEAs), which can stimulate only at fixed electrode locations. To characterize reliability and spatial selectivity of stimulation, we used intracellular (patch-clamp) recordings to monitor the effect of targeted laser pulses on cultured hippocampal neurons. Action potentials could be stimulated with submillisecond precision and 100-micron spatial resolution at rates exceeding 100 Hz. Optimal control parameters for stimulation are discussed.

Animals↗

PubMed2004

Noncontact measurement of nerve displacement during action potential with a dual-beam low-coherence interferometer.

We have used a novel phase-referenced heterodyne dual-beam low-coherence interferometer to perform what we believe are the first noncontact measurements of surface motion in a nerve bundle during the action potential. Nerve displacements of approximately 5-nm amplitude and approximately 10-ms duration are measured without signal averaging. This interferometer may find general application in measurement of small motion in cells and other weakly scattering samples.

Action Potentials↗

PubMed2004

Plasticity and tuning by visual feedback of the stability of a neural integrator.

Persistent neural firing is of fundamental importance to working memory and other brain functions because it allows information to be held "online" following an input and to be integrated over time. Many models of persistent activity rely on some kind of positive feedback internal to the neural circuit concerned; however, too much feedback causes runaway firing (instability), and too little results in loss of persistence (leak). This parameter sensitivity leads to the hypothesis that the brain uses an error signal (external feedback) to tune the stability of persistent firing by adjusting the amount of internal feedback. We test this hypothesis by manipulating external visual feedback, a putative sensory error signal, in a model system for persistent firing, the goldfish oculomotor neural integrator. Over tens of minutes to hours, electronically controlled visual feedback consistent with a leaky or unstable integrator can drive the integrator progressively more unstable or leaky, respectively. Eye fixation time constants can be reduced >100-fold to <1 s. Normal visual feedback gradually retunes the integrator back to stability. Changes in the phase of the sinusoidal vestibulo-ocular response are consistent with integrator detuning, as are changes in ocular drift following eye position shifts compensating for brief passive head movements during fixations. Corresponding changes in persistent firing of integrator neurons are presented in the accompanying article. The presence, strength, and reversibility of the plasticity demonstrate that, in this system, external visual feedback plays a vital role in gradually tuning the stability of the neural integrator.

Animals↗

PubMed2004

Plasticity and tuning of the time course of analog persistent firing in a neural integrator.

In a companion paper, we reported that the goldfish oculomotor neural integrator could be trained to instability or leak by rotating the visual surround with a velocity proportional to +/- horizontal eye position, respectively. Here we analyze changes in the firing rate behavior of neurons in area I in the caudal brainstem, a central component of the oculomotor neural integrator. Persistent firing could be detuned to instability and leak, respectively, along with fixation behavior. Prolonged training could reduce the time constant of persistent firing of some cells by more than an order of magnitude, to <1 s. Normal visual feedback gradually retuned persistent firing of integrator neurons toward stability, along with fixation behavior. In animals with unstable fixations, approximately half of the eye position-related cells had upward or unstable firing rate drift. In animals with leaky fixations, two-thirds of the eye position-related cells showed leaky firing drift. The remaining eye position-related cells, generally those with lower eye position thresholds, showed a more complex pattern of history-dependent/predictive firing rate drift in relation to eye drift. These complex drift cells often showed a drop in maximum persistent firing rate after training to leak. Despite this diversity, firing drift and the degree of instability or leak in firing rates were broadly correlated with fixation performance. The presence, strength, and reversibility of this plasticity demonstrate that, in this system, visual feedback plays a vital role in gradually tuning the time course of persistent neural firing.

Animals↗

PubMed2004

Learning in neural networks by reinforcement of irregular spiking.

Artificial neural networks are often trained by using the back propagation algorithm to compute the gradient of an objective function with respect to the synaptic strengths. For a biological neural network, such a gradient computation would be difficult to implement, because of the complex dynamics of intrinsic and synaptic conductances in neurons. Here we show that irregular spiking similar to that observed in biological neurons could be used as the basis for a learning rule that calculates a stochastic approximation to the gradient. The learning rule is derived based on a special class of model networks in which neurons fire spike trains with Poisson statistics. The learning is compatible with forms of synaptic dynamics such as short-term facilitation and depression. By correlating the fluctuations in irregular spiking with a reward signal, the learning rule performs stochastic gradient ascent on the expected reward. It is applied to two examples, learning the XOR computation and learning direction selectivity using depressing synapses. We also show in simulation that the learning rule is applicable to a network of noisy integrate-and-fire neurons.

Action Potentials↗

PubMed2004

Temporal sparseness of the premotor drive is important for rapid learning in a neural network model of birdsong.

Sparse neural codes have been widely observed in cortical sensory and motor areas. A striking example of sparse temporal coding is in the song-related premotor area high vocal center (HVC) of songbirds: The motor neurons innervating avian vocal muscles are driven by premotor nucleus robustus archistriatalis (RA), which is in turn driven by nucleus HVC. Recent experiments reveal that RA-projecting HVC neurons fire just one burst per song motif. However, the function of this remarkable temporal sparseness has remained unclear. Because birdsong is a clear example of a learned complex motor behavior, we explore in a neural network model with the help of numerical and analytical techniques the possible role of sparse premotor neural codes in song-related motor learning. In numerical simulations with nonlinear neurons, as HVC activity is made progressively less sparse, the minimum learning time increases significantly. Heuristically, this slowdown arises from increasing interference in the weight updates for different synapses. If activity in HVC is sparse, synaptic interference is reduced, and is minimized if each synapse from HVC to RA is used only once in the motif, which is the situation observed experimentally. Our numerical results are corroborated by a theoretical analysis of learning in linear networks, for which we derive a relationship between sparse activity, synaptic interference, and learning time. If songbirds acquire their songs under significant pressure to learn quickly, this study predicts that HVC activity, currently measured only in adults, should also be sparse during the sensorimotor phase in the juvenile bird. We discuss the relevance of these results, linking sparse codes and learning speed, to other multilayered sensory and motor systems.

Algorithms↗

PubMed2004

BCI Competition 2003--Data set Ia: combining gamma-band power with slow cortical potentials to improve single-trial classification of electroencephalographic signals.

In one type of brain-computer interface (BCI), users self-modulate brain activity as detected by electroencephalography (EEG). To infer user intent, EEG signals are classified by algorithms which typically use only one of the several types of information available in these signals. One such BCI uses slow cortical potential (SCP) measures to classify single trials. We complemented these measures with estimates of high-frequency (gamma-band) activity, which has been associated with attentional and intentional states. Using a simple linear classifier, we obtained significantly greater classification accuracy using both types of information from the same recording epochs compared to using SCPs alone.

Algorithms↗

PubMed2003

Learning in spiking neural networks by reinforcement of stochastic synaptic transmission.

It is well-known that chemical synaptic transmission is an unreliable process, but the function of such unreliability remains unclear. Here I consider the hypothesis that the randomness of synaptic transmission is harnessed by the brain for learning, in analogy to the way that genetic mutation is utilized by Darwinian evolution. This is possible if synapses are "hedonistic," responding to a global reward signal by increasing their probabilities of vesicle release or failure, depending on which action immediately preceded reward. Hedonistic synapses learn by computing a stochastic approximation to the gradient of the average reward. They are compatible with synaptic dynamics such as short-term facilitation and depression and with the intricacies of dendritic integration and action potential generation. A network of hedonistic synapses can be trained to perform a desired computation by administering reward appropriately, as illustrated here through numerical simulations of integrate-and-fire model neurons.

Action Potentials↗

PubMed2003

Correlated discharge among cell pairs within the oculomotor horizontal velocity-to-position integrator.

In the oculomotor system, temporal integration of velocity commands into position signals may depend on synaptic feedback among neurons of a bilateral brainstem cell assembly known as the "neural integrator." Both ipsilateral excitatory and contralateral inhibitory projections between eye position-related integrator cells are hypothesized as a substrate for positive feedback supporting integration. Presence of feedback interactions should be evident in cross-correlation functions of neuron pairs. Here, unilateral and bilateral paired recordings were obtained during fixation behavior from neurons in goldfish brainstem area I, a key element of the integrator. During fixations, discharge of most unilateral pairs, composed of cells with eye position sensitivities of the same sign, was positively correlated with lag of 0-10 msec (n = 11 of 14 significant). Typically, a very narrow peak (mean half-width <4 msec) near zero lag was observed. Discharge of bilateral pairs, composed of cells with position sensitivities of the opposite sign, was either negatively correlated with lag of 0-10 msec (n = 5 of 13 significant) or not correlated. Troughs in negative correlations always had minima between 3 and 5 msec lag. These results are consistent with the feedback hypothesis of temporal integration, highlighting excitation unilaterally and inhibition bilaterally. Absence of visual input did not weaken correlations, but other sources of correlated input extrinsic to area I were not ruled out. Triplet recordings revealed that unilateral pairwise correlations were primarily independent. Correlation between unilateral pairs systematically decreased with increasing eye position, demonstrating that synchrony is not necessary for persistent activity at high firing rates.

Action Potentials↗

PubMed2003

History dependence of rate covariation between neurons during persistent activity in an oculomotor integrator.

Persistent firing in response to a brief stimulus is a neural correlate of short-term memory in a variety of systems. In the oculomotor neural integrator, persistent firing that encodes eye position is maintained in response to transient saccadic eye-velocity commands. To a first approximation, firing rates in the integrator vary linearly with eye position. Thus, viewed across many cells, the pattern of persistent firing in the integrator may be constrained to a unique line of stable states. Here this idea was tested by examining the relationship between firing rates of simultaneously recorded neurons. Paired recordings were obtained in awake goldfish from neurons in hindbrain area I, an essential part of the horizontal eye-position integrator. During spontaneous eye movements consisting of sequential fixations at different horizontal positions, the pair relationship between the majority of cells on the same side of the integrator was not unique: for a given rate of one cell, the rate of the paired cell assumed different values that depended systematically on the preceding saccade history. This finding suggests that the set of persistent firing states that encode eye position is not constrained to a unique line, and that models with stable states restricted to a such a line need to be modified accordingly.

Action Potentials↗

PubMed2003

Permitted and forbidden sets in symmetric threshold-linear networks.

The richness and complexity of recurrent cortical circuits is an inexhaustible source of inspiration for thinking about high-level biological computation. In past theoretical studies, constraints on the synaptic connection patterns of threshold-linear networks were found that guaranteed bounded network dynamics, convergence to attractive fixed points, and multistability, all fundamental aspects of cortical information processing. However, these conditions were only sufficient, and it remained unclear which were the minimal (necessary) conditions for convergence and multistability. We show that symmetric threshold-linear networks converge to a set of attractive fixed points if and only if the network matrix is copositive. Furthermore, the set of attractive fixed points is nonconnected (the network is multiattractive) if and only if the network matrix is not positive semidefinite. There are permitted sets of neurons that can be coactive at a stable steady state and forbidden sets that cannot. Permitted sets are clustered in the sense that subsets of permitted sets are permitted and supersets of forbidden sets are forbidden. By viewing permitted sets as memories stored in the synaptic connections, we provide a formulation of long-term memory that is more general than the traditional perspective of fixed-point attractor networks. There is a close correspondence between threshold-linear networks and networks defined by the generalized Lotka-Volterra equations.

Animals↗

PubMed2003

Equivalence of backpropagation and contrastive Hebbian learning in a layered network.

Backpropagation and contrastive Hebbian learning are two methods of training networks with hidden neurons. Backpropagation computes an error signal for the output neurons and spreads it over the hidden neurons. Contrastive Hebbian learning involves clamping the output neurons at desired values and letting the effect spread through feedback connections over the entire network. To investigate the relationship between these two forms of learning, we consider a special case in which they are identical: a multilayer perceptron with linear output units, to which weak feedback connections have been added. In this case, the change in network state caused by clamping the output neurons turns out to be the same as the error signal spread by backpropagation, except for a scalar prefactor. This suggests that the functionality of backpropagation can be realized alternatively by a Hebbian-type learning algorithm, which is suitable for implementation in biological networks.

Algorithms↗