Biomedical subjects

John P O'Doherty

Publications and source records attributed to John P O'Doherty.

14 recordsLinked to original sources

PubMed2006

Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems.

When deciding between different options, individuals are guided by the expected (mean) value of the different outcomes and by the associated degrees of uncertainty. We used functional magnetic resonance imaging to identify brain activations coding the key decision parameters of expected value (magnitude and probability) separately from uncertainty (statistical variance) of monetary rewards. Participants discriminated behaviorally between stimuli associated with different expected values and uncertainty. Stimuli associated with higher expected values elicited monotonically increasing activations in distinct regions of the striatum, irrespective of different combinations of magnitude and probability. Stimuli associated with higher uncertainty (variance) elicited increasing activations in the lateral orbitofrontal cortex. Uncertainty-related activations covaried with individual risk aversion in lateral orbitofrontal regions and risk-seeking in more medial areas. Furthermore, activations in expected value-coding regions in prefrontal cortex covaried differentially with uncertainty depending on risk attitudes of individual participants, suggesting that separate prefrontal regions are involved in risk aversion and seeking. These data demonstrate the distinct coding in key reward structures of the two basic and crucial decision parameters, expected value, and uncertainty.

Adult↗

PubMed2006

The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans.

Many real-life decision-making problems incorporate higher-order structure, involving interdependencies between different stimuli, actions, and subsequent rewards. It is not known whether brain regions implicated in decision making, such as the ventromedial prefrontal cortex (vmPFC), use a stored model of the task structure to guide choice (model-based decision making) or merely learn action or state values without assuming higher-order structure as in standard reinforcement learning. To discriminate between these possibilities, we scanned human subjects with functional magnetic resonance imaging while they performed a simple decision-making task with higher-order structure, probabilistic reversal learning. We found that neural activity in a key decision-making region, the vmPFC, was more consistent with a computational model that exploits higher-order structure than with simple reinforcement learning. These results suggest that brain regions, such as the vmPFC, use an abstract model of task structure to guide behavioral choice, computations that may underlie the human capacity for complex social interactions and abstract strategizing.

Action Potentials↗

PubMed2006

Cortical substrates for exploratory decisions in humans.

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.

Choice Behavior↗

PubMed2006

Empathic neural responses are modulated by the perceived fairness of others.

The neural processes underlying empathy are a subject of intense interest within the social neurosciences. However, very little is known about how brain empathic responses are modulated by the affective link between individuals. We show here that empathic responses are modulated by learned preferences, a result consistent with economic models of social preferences. We engaged male and female volunteers in an economic game, in which two confederates played fairly or unfairly, and then measured brain activity with functional magnetic resonance imaging while these same volunteers observed the confederates receiving pain. Both sexes exhibited empathy-related activation in pain-related brain areas (fronto-insular and anterior cingulate cortices) towards fair players. However, these empathy-related responses were significantly reduced in males when observing an unfair person receiving pain. This effect was accompanied by increased activation in reward-related areas, correlated with an expressed desire for revenge. We conclude that in men (at least) empathic responses are shaped by valuation of other people's social behaviour, such that they empathize with fair opponents while favouring the physical punishment of unfair opponents, a finding that echoes recent evidence for altruistic punishment.

Brain↗

PubMed2006

Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum.

Food preferences are acquired through experience and can exert strong influence on choice behavior. In order to choose which food to consume, it is necessary to maintain a predictive representation of the subjective value of the associated food stimulus. Here, we explore the neural mechanisms by which such predictive representations are learned through classical conditioning. Human subjects were scanned using fMRI while learning associations between arbitrary visual stimuli and subsequent delivery of one of five different food flavors. Using a temporal difference algorithm to model learning, we found predictive responses in the ventral midbrain and a part of ventral striatum (ventral putamen) that were related directly to subjects' actual behavioral preferences. These brain structures demonstrated divergent response profiles, with the ventral midbrain showing a linear response profile with preference, and the ventral striatum a bivalent response. These results provide insight into the neural mechanisms underlying human preference behavior.

Adult↗

PubMed2006

Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain.

Avoidance learning poses a challenge for reinforcement-based theories of instrumental conditioning, because once an aversive outcome is successfully avoided an individual may no longer experience extrinsic reinforcement for their behavior. One possible account for this is to propose that avoiding an aversive outcome is in itself a reward, and thus avoidance behavior is positively reinforced on each trial when the aversive outcome is successfully avoided. In the present study we aimed to test this possibility by determining whether avoidance of an aversive outcome recruits the same neural circuitry as that elicited by a reward itself. We scanned 16 human participants with functional MRI while they performed an instrumental choice task, in which on each trial they chose from one of two actions in order to either win money or else avoid losing money. Neural activity in a region previously implicated in encoding stimulus reward value, the medial orbitofrontal cortex, was found to increase, not only following receipt of reward, but also following successful avoidance of an aversive outcome. This neural signal may itself act as an intrinsic reward, thereby serving to reinforce actions during instrumental avoidance.

Adult↗

PubMed2005

Contingency awareness in human aversive conditioning involves the middle frontal gyrus.

In contrast to the wealth of data describing the neural mechanisms underlying classical conditioning, we know remarkably little about the mechanisms involved in acquisition of explicit contingency awareness. Subjects variably acquire contingency awareness in classical conditioning paradigms, in which they are able to describe the temporal relationship between a conditioned cue and its outcome. Previous studies have implicated the hippocampus and prefrontal cortex in the acquisition of explicit knowledge, although their specific roles remain unclear. We used functional magnetic resonance imaging to track the trial-by-trial acquisition of explicit knowledge in a concurrent trace and delay conditioning paradigm. We show that activity in bilateral middle frontal gyrus and parahippocampal gyrus correlates with the accuracy of explicit contingency awareness on each trial. In contrast, amygdala activation correlates with conditioned responses indexed by skin conductance responses (SCRs). These results demonstrate that brain regions known to be involved in other aspects of learning and memory also play a specific role, reflecting on each trial the acquisition and representation of contingency awareness.

Adult↗

PubMed2005

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

Avoidance Learning↗

PubMed2005

Regret and its avoidance: a neuroimaging study of choice behavior.

Human decisions can be shaped by predictions of emotions that ensue after choosing advantageously or disadvantageously. Indeed, anticipating regret is a powerful predictor of future choices. We measured brain activity using functional magnetic resonance imaging (fMRI) while subjects selected between two gambles wherein regret was induced by providing information about the outcome of the unchosen gamble. Increasing regret enhanced activity in the medial orbitofrontal region, the anterior cingulate cortex and the hippocampus. Notably, across the experiment, subjects became increasingly regret-aversive, a cumulative effect reflected in enhanced activity within medial orbitofrontal cortex and amygdala. This pattern of activity reoccurred just before making a choice, suggesting that the same neural circuitry mediates direct experience of regret and its anticipation. These results demonstrate that medial orbitofrontal cortex modulates the gain of adaptive emotions in a manner that may provide a substrate for the influence of high-level emotions on decision making.

Adult↗

PubMed2005

Anxiety reduction through detachment: subjective, physiological, and neural effects.

The ability to volitionally regulate emotions helps to adapt behavior to changing environmental demands and can alleviate subjective distress. We show that a cognitive strategy of detachment attenuates subjective and physiological measures of anticipatory anxiety for pain and reduces reactivity to receipt of pain itself. Using functional magnetic resonance imaging, we locate the potential site and source of this modulation of anticipatory anxiety in the medial prefrontal/anterior cingulate and anterolateral prefrontal cortex, respectively.

Adaptation, Psychological↗

PubMed2004

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Conditioning, Classical↗

PubMed2004

Reward representations and reward-related learning in the human brain: insights from neuroimaging.

This review outlines recent findings from human neuroimaging concerning the role of a highly interconnected network of brain areas including orbital and medial prefrontal cortex, amygdala, striatum and dopaminergic mid-brain in reward processing. Distinct reward-related functions can be attributed to different components of this network. Orbitofrontal cortex is involved in coding stimulus reward value and in concert with the amygdala and ventral striatum is implicated in representing predicted future reward. Such representations can be used to guide action selection for reward, a process that depends, at least in part, on orbital and medial prefrontal cortex as well as dorsal striatum.

Animals↗

PubMed2003

Temporal difference models and reward-related learning in the human brain.

Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.

Adolescent↗

PubMed2002

Neural responses during anticipation of a primary taste reward.

The aim of this study was to determine the brain regions involved in anticipation of a primary taste reward and to compare these regions to those responding to the receipt of a taste reward. Using fMRI, we scanned human subjects who were presented with visual cues that signaled subsequent reinforcement with a pleasant sweet taste (1 M glucose), a moderately unpleasant salt taste (0.2 M saline), or a neutral taste. Expectation of a pleasant taste produced activation in dopaminergic midbrain, posterior dorsal amygdala, striatum, and orbitofrontal cortex (OFC). Apart from OFC, these regions were not activated by reward receipt. The findings indicate that when rewards are predictable, brain regions recruited during expectation are, in part, dissociable from areas responding to reward receipt.

Adolescent↗