PubMed Health⌕ Search

Biomedical subjects

Wolfram Schultz

Publications and source records attributed to Wolfram Schultz.

15 recordsLinked to original sources
PubMed2006

Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems.

When deciding between different options, individuals are guided by the expected (mean) value of the different outcomes and by the associated degrees of uncertainty. We used functional magnetic resonance imaging to identify brain activations coding the key decision parameters of expected value (magnitude and probability) separately from uncertainty (statistical variance) of monetary rewards. Participants discriminated behaviorally between stimuli associated with different expected values and uncertainty. Stimuli associated with higher expected values elicited monotonically increasing activations in distinct regions of the striatum, irrespective of different combinations of magnitude and probability. Stimuli associated with higher uncertainty (variance) elicited increasing activations in the lateral orbitofrontal cortex. Uncertainty-related activations covaried with individual risk aversion in lateral orbitofrontal regions and risk-seeking in more medial areas. Furthermore, activations in expected value-coding regions in prefrontal cortex covaried differentially with uncertainty depending on risk attitudes of individual participants, suggesting that separate prefrontal regions are involved in risk aversion and seeking. These data demonstrate the distinct coding in key reward structures of the two basic and crucial decision parameters, expected value, and uncertainty.

Adult↗
PubMed2006

Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex.

Both appetitive and aversive outcomes can reinforce animal behavior. It is not clear, however, whether the opposing kinds of reinforcers are processed by specific or common neural mechanisms. To investigate this issue, we studied macaque monkeys that performed a memory-guided saccade task for three different outcomes, namely delivery of liquid reward, avoidance of air puff, and feedback sound only. Animals performed the task best in rewarded trials, intermediately in aversive trials, and worst in sound-only trials. Most task-related activity in lateral prefrontal cortex was differentially influenced by the reinforcers. Aversive avoidance had clear effects on some prefrontal neurons, although the effects of rewards were more common. We also observed neurons modulated by both positive and negative reinforcers, reflecting reinforcement or attentional processes. Our results demonstrate that information about positive and negative reinforcers is processed differentially in prefrontal cortex, which could contribute to the role of this structure in goal-directed behavior.

Analysis of Variance↗
PubMed2006

Behavioral theories and the neurophysiology of reward.

The functions of rewards are based primarily on their effects on behavior and are less directly governed by the physics and chemistry of input events as in sensory systems. Therefore, the investigation of neural mechanisms underlying reward functions requires behavioral theories that can conceptualize the different effects of rewards on behavior. The scientific investigation of behavioral processes by animal learning theory and economic utility theory has produced a theoretical framework that can help to elucidate the neural correlates for reward functions in learning, goal-directed approach behavior, and decision making under uncertainty. Individual neurons can be studied in the reward systems of the brain, including dopamine neurons, orbitofrontal cortex, and striatum. The neural activity can be related to basic theoretical terms of reward and uncertainty, such as contiguity, contingency, prediction error, magnitude, probability, expected value, and variance.

Animals↗
PubMed2005

Human neural learning depends on reward prediction errors in the blocking paradigm.

Learning occurs when an outcome deviates from expectation (prediction error). According to formal learning theory, the defining paradigm demonstrating the role of prediction errors in learning is the blocking test. Here, a novel stimulus is blocked from learning when it is associated with a fully predicted outcome, presumably because the occurrence of the outcome fails to produce a prediction error. We investigated the role of prediction errors in human reward-directed learning using a blocking paradigm and measured brain activation with functional magnetic resonance imaging. Participants showed blocking of behavioral learning with juice rewards as predicted by learning theory. The medial orbitofrontal cortex and the ventral putamen showed significantly lower responses to blocked, compared with nonblocked, reward-predicting stimuli. In reward-predicting control situations, deactivation in orbitofrontal cortex and ventral putamen occurred at the time of unpredicted reward omissions. Responses in discrete parts of orbitofrontal cortex correlated with the degree of behavioral learning during, and after, the learning phase. These data suggest that learning in primary reward structures in the human brain correlates with prediction errors in a manner that complies with principles of formal learning theory.

Adult↗
PubMed2005

Rewarding properties of visual stimuli.

The behavioral functions of rewards comprise the induction of learning and approach behavior. Rewards are not only related to vegetative states of hunger, thirst and reproduction but may also consist of visual stimuli. The present experiment tested the reward potential of different types of still and moving pictures in three operant tasks involving key press, touch of computer monitor and choice behavior in a laboratory environment. We found that all tested visual stimuli induced approach behavior in all three tasks, and that action movies sustained consistently higher rates of responding compared to changing still pictures, which were more effective than constant still pictures. These results demonstrate that visual stimuli can serve as positive reinforcers for operant reactions of animals in controlled laboratory settings. In particular, the coherently animated visual stimuli of movies have considerable reward potential. These observations would allow similar forms of visual rewards to be used for neurophysiological investigations of mechanisms related to non-vegetative rewards.

Animals↗
PubMed2005

Adaptive coding of reward value by dopamine neurons.

It is important for animals to estimate the value of rewards as accurately as possible. Because the number of potential reward values is very large, it is necessary that the brain's limited resources be allocated so as to discriminate better among more likely reward outcomes at the expense of less likely outcomes. We found that midbrain dopamine neurons rapidly adapted to the information provided by reward-predicting stimuli. Responses shifted relative to the expected reward value, and the gain adjusted to the variance of reward value. In this way, dopamine neurons maintained their reward sensitivity over a large range of reward values.

Adaptation, Physiological↗
PubMed2005

Relative reward processing in primate striatum.

Rewards are often not only valued according to their physical characteristics but also relative to other available rewards. The striatum (caudate nucleus, putamen, ventral striatum including nucleus accumbens) is involved in the organization of movement and the processing of reward information. We studied the activity of single striatal neurons in macaques that were presented with different combinations of two rewards. We found in nearly half of the investigated neurons that the processing for one reward shifted, relative to the other rewards that were available in a given trial block. The relative reward processing concerned all forms of striatal activity related to reward-predicting visual stimuli, arm movements and reception of rewards. The observed changes may provide a neural basis for the known shifts in valuation of rewarding outcomes relative to known references.

Animals↗
PubMed2004

Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology.

Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.

Animals↗
PubMed2003

Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm.

Animals learn not only about stimuli that predict reward but also about those that signal the omission of an expected reward. We used a conditioned inhibition paradigm derived from animal learning theory to train a discrimination between a visual stimulus that predicted reward (conditioned excitor) and a second stimulus that predicted the omission of reward (conditioned inhibitor). Performing the discrimination required attention to both the conditioned excitor and the inhibitor; however, dopamine neurons showed very different responses to the two classes of stimuli. Conditioned inhibitors elicited considerable depressions in 48 of 69 neurons (median of 35% below baseline) and minor activations in 29 of 69 neurons (69% above baseline), whereas reward-predicting excitors induced pure activations in all 69 neurons tested (242% above baseline), thereby demonstrating that the neurons discriminated between conditioned stimuli predicting reward versus nonreward. The discriminative responses to stimuli with differential reward-predicting but common attentional functions indicate differential neural coding of reward prediction and attention. The neuronal responses appear to reflect reward prediction errors, thus suggesting an extension of the correspondence between learning theory and activity of single dopamine neurons to the prediction of nonreward.

Animals↗
PubMed2003

Discrete coding of reward probability and uncertainty by dopamine neurons.

Uncertainty is critical in the measure of information and in assessing the accuracy of predictions. It is determined by probability P, being maximal at P = 0.5 and decreasing at higher and lower probabilities. Using distinct stimuli to indicate the probability of reward, we found that the phasic activation of dopamine neurons varied monotonically across the full range of probabilities, supporting past claims that this response codes the discrepancy between predicted and actual reward. In contrast, a previously unobserved response covaried with uncertainty and consisted of a gradual increase in activity until the potential time of reward. The coding of uncertainty suggests a possible role for dopamine signals in attention-based learning and risk-taking behavior.

Animals↗
PubMed2003

Effects of expectations for different reward magnitudes on neuronal activity in primate striatum.

In behavioral science, it is well known that humans and nonhuman animals are highly sensitive to differences in reward magnitude when choosing an outcome from a set of alternatives. We know that a realm of behavioral reactions is altered when animals begin to expect different levels of reward outcome. Our present aim was to investigate how the expectation for different magnitudes of reward influences behavior-related neurophysiology in the anterior striatum. In a spatial delayed response task, different instruction pictures are presented to the monkey. Each image represents a different magnitude of juice. By reaching to the spatial location where an instruction picture was presented, animals could receive the particular liquid amount designated by the stimulus. Reliable preferences in reward choice trials and differences in anticipatory licks, performance errors, and reaction times indicated that animals differentially expected the various reward amounts predicted by the instruction cues. A total of 374 of 2,000 neurons in the anterior parts of the caudate nucleus, putamen, and ventral striatum showed five forms of task-related activation during the preparation or execution of movement and activations preceding or following the liquid drop delivery. Approximately one-half of these striatal neurons showed differing response levels dependent on the magnitude of liquid to be received. Results of a linear regression analysis showed that reward magnitude and single cell discharge rate were related in a subset of neurons by a monotonic positive or negative relationship. Overall, these data support the idea that the striatum utilizes expectancies that contain precise information concerning the predicted, forthcoming level of reward in directing general behavioral reactions.

Animals↗
PubMed2003

Changes in behavior-related neuronal activity in the striatum during learning.

The involvement of the striatum in numerous forms of learning and memory is likely to be based on changes in neuronal activity when specific behavioral tasks are being learned. Striatal neurons show distinctive changes when animals learn the significance of stimuli that predict rewards and induce the preparation of movements. These changes resemble some of the simultaneous, learning-related changes in closely associated areas of the frontal cortex. The striatal changes might assist in adapting existing reward expectations and behaviors to novel or changing environmental conditions and they could contribute to the functions of the basal ganglia in learning, reward expectation and movement preparation.

Animals↗
PubMed2002

Getting formal with dopamine and reward.

Recent neurophysiological studies reveal that neurons in certain brain structures carry specific signals about past and future rewards. Dopamine neurons display a short-latency, phasic reward signal indicating the difference between actual and predicted rewards. The signal is useful for enhancing neuronal processing and learning behavioral reactions. It is distinctly different from dopamine's tonic enabling of numerous behavioral processes. Neurons in the striatum, frontal cortex, and amygdala also process reward information but provide more differentiated information for identifying and anticipating rewards and organizing goal-directed behavior. The different reward signals have complementary functions, and the optimal use of rewards in voluntary behavior would benefit from interactions between the signals. Addictive psychostimulant drugs may exert their action by amplifying the dopamine reward signal.

Animals↗
PubMed1999

The Reward Signal of Midbrain Dopamine Neurons.

Dopamine projections from the midbrain to striatum and frontal cortex play a major role in behavioral reactions controlled by rewards. Recent experiments have shown that dopamine neurons code the discrepancy between the prediction and occurrence of rewards and in this way signal a crucial learning term for approach behavior.

Journal Article↗