Biomedical subjects

David S Touretzky

Publications and source records attributed to David S Touretzky.

6 recordsLinked to original sources

PubMed2006

Bayesian theories of conditioning in a changing world.

The recent flowering of Bayesian approaches invites the re-examination of classic issues in behavior, even in areas as venerable as Pavlovian conditioning. A statistical account can offer a new, principled interpretation of behavior, and previous experiments and theories can inform many unexplored aspects of the Bayesian enterprise. Here we consider one such issue: the finding that surprising events provoke animals to learn faster. We suggest that, in a statistical account of conditioning, surprise signals change and therefore uncertainty and the need for new learning. We discuss inference in a world that changes and show how experimental results involving surprise can be interpreted from this perspective, and also how, thus understood, these phenomena help constrain statistical theories of animal and human learning.

Animals↗

PubMed2006

A spin glass model of path integration in rat medial entorhinal cortex.

Electrophysiological recording studies in the dorsocaudal region of medial entorhinal cortex (dMEC) of the rat reveal cells whose spatial firing fields show a remarkably regular hexagonal grid pattern (Fyhn et al., 2004; Hafting et al., 2005). We describe a symmetric, locally connected neural network, or spin glass model, that spontaneously produces a hexagonal grid of activity bumps on a two-dimensional sheet of units. The spatial firing fields of the simulated cells closely resemble those of dMEC cells. A collection of grids with different scales and/or orientations forms a basis set for encoding position. Simulations show that the animal's location can easily be determined from the population activity pattern. Introducing an asymmetry in the model allows the activity bumps to be shifted in any direction, at a rate proportional to velocity, to achieve path integration. Furthermore, information about the structure of the environment can be superimposed on the spatial position signal by modulation of the bump activity levels without significantly interfering with the hexagonal periodicity of firing fields. Our results support the conjecture of Hafting et al. (2005) that an attractor network in dMEC may be the source of path integration information afferent to hippocampus.

Animals↗

PubMed2006

Representation and timing in theories of the dopamine system.

Although the responses of dopamine neurons in the primate midbrain are well characterized as carrying a temporal difference (TD) error signal for reward prediction, existing theories do not offer a credible account of how the brain keeps track of past sensory events that may be relevant to predicting future reward. Empirically, these shortcomings of previous theories are particularly evident in their account of experiments in which animals were exposed to variation in the timing of events. The original theories mispredicted the results of such experiments due to their use of a representational device called a tapped delay line. Here we propose that a richer understanding of history representation and a better account of these experiments can be given by considering TD algorithms for a formal setting that incorporates two features not originally considered in theories of the dopaminergic response: partial observability (a distinction between the animal's sensory experience and the true underlying state of the world) and semi-Markov dynamics (an explicit account of variation in the intervals between events). The new theory situates the dopaminergic system in a richer functional and anatomical context, since it assumes (in accord with recent computational theories of cortex) that problems of partial observability and stimulus history are solved in sensory cortex using statistical modeling and inference and that the TD system predicts reward using the results of this inference rather than raw sensory data. It also accounts for a range of experimental data, including the experiments involving programmed temporal variability and other previously unmodeled dopaminergic response phenomena, which we suggest are related to subjective noise in animals' interval timing. Finally, it offers new experimental predictions and a rich theoretical framework for designing future experiments.

Algorithms↗

PubMed2005

Influence of path integration versus environmental orientation on place cell remapping between visually identical environments.

To assess the effects of interactions between angular path integration and visual landmarks on the firing of hippocampal neurons, we recorded from CA1 pyramidal cells as rats foraged in two identical boxes with polarizing internal cues. In the same-orientation condition, following an earlier experiment by Skaggs and McNaughton, the boxes were oriented identically and connected by a corridor. In the opposite-orientation condition, the boxes were abutted by rotating them 90 degrees in opposite directions, so that their orientations differed by 180 degrees . After 16-23 days of pretraining on the same-orientation condition, three rats experienced both conditions in counterbalanced order on each of two consecutive days. On the third day they ran two opposite-orientation trials. Although Skaggs and McNaughton observed stable partial "remapping" of place fields, none of the fields in this experiment remapped in the same-orientation condition. In the opposite-orientation condition, place fields in the first box were isomorphic with those in the same-orientation condition, whereas in the second box the rats eventually exhibited completely different fields. The rats differed as to the trial in which this first occurred. Once the second box exhibited different fields, it continued to do so in all subsequent opposite-orientation trials, yet fields remained the same in subsequent same-orientation trials. The results demonstrate that when animals move actively between environments, and are thus potentially able to maintain their inertial angular orientation, discordance between environmental orientation and the rat's idiothetic direction sense can profoundly affect the hippocampal map-either immediately, or as a result of cumulative experience.

Analysis of Variance↗

PubMed2005

Deforming the hippocampal map.

To investigate conjoint stimulus control over place cells, Fenton et al. (J Gen Physiol 116:191-209, 2000a) recorded while rats foraged in a cylinder with 45 degrees black and white cue cards on the wall. Card centers were 135 degrees apart. In probe trials, the cards were rotated together or apart by 25 degrees . Firing field centers shifted during these trials, stretching and shrinking the cognitive map. Fenton et al. (2000b) described this deformation with an ad hoc vector field equation. We consider what sorts of neural network mechanisms might be capable of accounting for their observations. In an abstract, maximum likelihood formulation, the rat's location is estimated by a conjoint probability density function of landmark positions. In an attractor neural network model, recurrent connections produce a bump of activity over a two-dimensional array of cells; the bump's position is influenced by landmark features such as distances or bearings. If features are chosen with appropriate care, the attractor network and maximum likelihood models yield similar results, in accord with previous demonstrations that recurrent neural networks can efficiently implement maximum likelihood computations (Pouget et al. Neural Comput 10:373-401, 1998; Deneve et al. Nat Neurosci 4:826-831, 2001).

Action Potentials↗

PubMed2002

Long-term reward prediction in TD models of the dopamine system.

This article addresses the relationship between long-term reward predictions and slow-timescale neural activity in temporal difference (TD) models of the dopamine system. Such models attempt to explain how the activity of dopamine (DA) neurons relates to errors in the prediction of future rewards. Previous models have been mostly restricted to short-term predictions of rewards expected during a single, somewhat artificially defined trial. Also, the models focused exclusively on the phasic pause-and-burst activity of primate DA neurons; the neurons' slower, tonic background activity was assumed to be constant. This has led to difficulty in explaining the results of neurochemical experiments that measure indications of DA release on a slow timescale, results that seem at first glance inconsistent with a reward prediction model. In this article, we investigate a TD model of DA activity modified so as to enable it to make longer-term predictions about rewards expected far in the future. We show that these predictions manifest themselves as slow changes in the baseline error signal, which we associate with tonic DA activity. Using this model, we make new predictions about the behavior of the DA system in a number of experimental situations. Some of these predictions suggest new computational explanations for previously puzzling data, such as indications from microdialysis studies of elevated DA activity triggered by aversive events.

Animals↗