The phasic reward signal of primate dopamine neurons.
Explore the source record for details and available documents.
Animals↗
Biomedical subjects
W Schultz
Publications and source records attributed to W Schultz.
Dopamine neurons report an error in the temporal prediction of reward during learning.
Many behaviors are affected by rewards, undergoing long-term changes when rewards are different than predicted but remaining unchanged when rewards occur exactly as predicted. The discrepancy between reward occurrence and reward prediction is termed an 'error in reward prediction'. Dopamine neurons in the substantia nigra and the ventral tegmental area are believed to be involved in reward-dependent behaviors. Consistent with this role, they are activated by rewards, and because they are activated more strongly by unpredicted than by predicted rewards they may play a role in learning. The present study investigated whether monkey dopamine neurons code an error in reward prediction during the course of learning. Dopamine neuron responses reflected the changes in reward prediction during individual learning episodes; dopamine neurons were activated by rewards during early trials, when errors were frequent and rewards unpredictable, but activation was progressively reduced as performance was consolidated and rewards became more predictable. These neurons were also activated when rewards occurred at unpredicted times and were depressed when rewards were omitted at the predicted times. Thus, dopamine neurons code errors in the prediction of both the occurrence and the time of rewards. In this respect, their responses resemble the teaching signals that have been employed in particularly efficient computational learning models.
Animals↗
Light and electron microscopic in-situ hybridization of collagen type I and type II mRNA in the fibrocartilaginous tissue of late-stage osteoarthritis.
OBJECTIVE: Biochemical analysis indicates the presence of collagen type I in fibrocartilaginous tissue of osteoarthritic cartilage, whereas normal hyaline cartilage contains only collagen type II produced by normal chondrocytes. Fibrocartilaginous tissue of late-stage osteoarthritis also exhibits irregularly shaped type 2b secretory chondrocytes as described in the literature. We have attempted to elucidate the type of cell which produces each type of collagen in late-stage osteoarthritis. DESIGN: We carried out in-situ hybridization at the light and electron microscopic level on the same tissue embedded in LR-Gold applying silver enhancement for gold-coupled anti-DIG antibodies. We correlated the types of cells with the expression of transcripts for type I and type II collagen. RESULTS: We found that cells resembling type 2b secretory chondrocytes of deep zones of fibrocartilaginous tissue expressed collagen type I mRNA and almost no collagen type II mRNA. The amount of collagen type I mRNA was as high as the amount produced in normal human skin fibroblasts. CONCLUSION: Some of the collagen type I in osteoarthritic human cartilage of late-stage disease is produced by cells resembling type 2b secretory chondrocytes of the deep zone.
Adult↗
Predictive reward signal of dopamine neurons.
The effects of lesions, receptor blocking, electrical self-stimulation, and drugs of abuse suggest that midbrain dopamine systems are involved in processing reward information and learning approach behavior. Most dopamine neurons show phasic activations after primary liquid and food rewards and conditioned, reward-predicting visual and auditory stimuli. They show biphasic, activation-depression responses after stimuli that resemble reward-predicting stimuli or are novel or particularly salient. However, only few phasic activations follow aversive stimuli. Thus dopamine neurons label environmental stimuli with appetitive value, predict and detect rewards and signal alerting and motivating events. By failing to discriminate between different rewards, dopamine neurons appear to emit an alerting message about the surprising presence or absence of rewards. All responses to rewards and reward-predicting stimuli depend on event predictability. Dopamine neurons are activated by rewarding events that are better than predicted, remain uninfluenced by events that are as good as predicted, and are depressed by events that are worse than predicted. By signaling rewards according to a prediction error, dopamine responses have the formal characteristics of a teaching signal postulated by reinforcement learning theories. Dopamine responses transfer during learning from primary rewards to reward-predicting stimuli. This may contribute to neuronal mechanisms underlying the retrograde action of rewards, one of the main puzzles in reinforcement learning. The impulse response releases a short pulse of dopamine onto many dendrites, thus broadcasting a rather global reinforcement signal to postsynaptic neurons. This signal may improve approach behavior by providing advance reward information before the behavior occurs, and may contribute to learning by modifying synaptic transmission. The dopamine reward signal is supplemented by activity in neurons in striatum, frontal cortex, and amygdala, which process specific reward information but do not emit a global reward prediction error signal. A cooperation between the different reward signals may assure the use of specific rewards for selectively reinforcing behaviors. Among the other projection systems, noradrenaline neurons predominantly serve attentional mechanisms and nucleus basalis neurons code rewards heterogeneously. Cerebellar climbing fibers signal errors in motor performance or errors in the prediction of aversive events to cerebellar Purkinje cells. Most deficits following dopamine-depleting lesions are not easily explained by a defective reward signal but may reflect the absence of a general enabling function of tonic levels of extracellular dopamine. Thus dopamine systems may have two functions, the phasic transmission of reward information and the tonic enabling of postsynaptic neurons.
Animals↗
Influence of reward expectation on behavior-related neuronal activity in primate striatum.
Rewards constitute important goals for voluntary behavior. This study aimed to investigate how expected rewards influence behavior-related neuronal activity in the anterior striatum. In a delayed go-nogo task, monkeys executed or withheld a reaching movement and obtained liquid or sound as reinforcement. An initial instruction picture indicated the behavioral reaction to be performed and the reinforcer to be obtained after a subsequent trigger stimulus. Movements varied according to the reinforcers predicted by the instructions, suggesting that animals differentially expected the two outcomes. About 250 of nearly 1,500 neurons in anterior parts of caudate nucleus, putamen, and ventral striatum showed typical task-related activations that reflected the expectation of instructions and trigger, and the preparation, initiation, and execution of behavioral reactions. Strikingly, most task-related activations occurred only when liquid reward was delivered at trial end, rather than the reinforcing sound. Activations close to the time of reward showed similar preferences for liquid reward over the reinforcing sound, suggesting a relationship to the expectation or detection of the motivational outcome of the trial rather than to a "correct" or "end-of-trial" signal. By contrast, relatively few activations in the present task occurred irrespective of the type of reinforcement. In conclusion, many of the behavior-related neurons investigated in the anterior striatum were influenced by an upcoming primary liquid reward and did not appear to code behavioral acts in a motivationally neutral manner. Rather, these neurons incorporated information about the expected outcome into their behavior-related activity. The activations influenced by reward several seconds before its occurrence may constitute a neuronal basis for the retrograde effects of rewards on behavioral reactions.
Animals↗
Modifications of reward expectation-related neuronal activity during learning in primate striatum.
This study investigated neuronal activity in the anterior striatum while monkeys repeatedly learned to associate new instruction stimuli with known behavioral reactions and reinforcers. In a delayed go-nogo task with several trial types, an initial picture instructed the animal to execute or withhold a reaching movement and to expect a liquid reward or not. During learning, new instruction pictures were presented, and animals guessed and performed one of the trial types according to a trial-and-error strategy. Learning of a large number of pictures resulted in a learning set in which learning took place in a few trials and correct performance exceeded 80% in the first 60-90 trials. About 200 task-related striatal neurons studied in both familiar and learning conditions showed three forms of changes during learning. Activations related to the preparation and execution of behavioral reactions and the expectation of reward were maintained in many neurons but occurred in inappropriate trial types when behavioral errors were made. The activations became appropriate for individual trial types when the animals' behavior adapted to the new task contingencies. In particular, reward expectation-related activations occurred initially in both rewarded and unrewarded movement trials and became subsequently restricted to rewarded trials. These changes occurred in parallel with the visible adaptation of reward expectations by the animals. The second learning change consisted in decreases of task-related activations that were either restricted to the initial trials of new learning problems or persisted during the subsequent consolidation phase. They probably reflected reductions in the expectation and preparation of upcoming task events, including reward. The third learning change consisted in transient or sustained increases of activations. These might reflect the increased attention accompanying learning and serve to induce synaptic changes underlying the behavioral adaptations. Both decreases and increases often induced changes in the trial selective occurrence of activations. In conclusion, neurons in anterior striatum showed changes related to adaptations or reductions of expectations in new task situations and displayed activations that might serve to induce structural changes during learning.
Animals↗
Activation of the human brain by monetary reward.
With the purpose of studying neural activation associated with reward processing in humans, we measured regional cerebral blood flow in 10 right-handed healthy subjects performing a delayed go-no go task in two different reinforcement conditions. Correct responses were either rewarded by money or a simple "ok' reinforcer. Behaviour rewarded by money, as compared with the "ok' reinforcement, was most significantly associated with activation of dorsolateral and orbital frontal cortex and also involved the midbrain and thalamus. These results may reflect the processing of reward information, although arousal effects cannot be completely excluded. It is suggested that the observed foci are implicated in the assessment of consequences in goal-directed behaviour which agrees with research in non-human primates.
Adult↗
A neural substrate of prediction and reward.
The capacity to predict future events permits a creature to detect, model, and manipulate the causal structure of its interactions with its environment. Behavioral experiments suggest that learning is driven by changes in the expectations about future salient events such as rewards and punishments. Physiological work has recently complemented these studies by identifying dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events. Taken together, these findings can be understood through quantitative theories of adaptive optimizing control.
Algorithms↗
Dopamine neurons and their role in reward mechanisms.
Information related to rewards is processed by a limited number of brain structures. Recent studies have demonstrated that dopamine neurons respond to appetitive events, such as primary rewards and reward-predicting stimuli. Rather than responding unconditionally, these neurons signal deviations from the prediction of future appetitive events. These reward-related responses correspond formally to concepts of behavioral and computational learning theories and may thus constitute teaching signals for appetitive learning.
Animals↗
Systemic effects of open and arthroscopic articulosynovectomy compared to radiosynoviorthesis in patients with rheumatoid arthritis.
Explore the source record for details and available documents.
Adult↗
Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli.
Midbrain dopamine systems are crucially involved in motivational processes underlying the learning and execution of goal-directed behaviour. Dopamine neurons in monkeys are uniformly activated by unpredicted appetitive stimuli such as food and liquid rewards and conditioned, reward-predicting stimuli. By contrast, fully predicted stimuli are ineffective, and the omission of predicted reward depresses their activity. These characteristics follow associative-learning rules, suggesting that dopamine responses report an error in reward prediction. Accordingly, neural network models are efficiently trained using a dopamine-like reinforcement signal. However, it is unknown whether the responses to environmental stimuli concern specific motivational attributes or reflect more general stimulus salience. To resolve this, we have compared dopamine impulse responses to motivationally opposing appetitive and aversive stimuli. In contrast to appetitive events, primary and conditioned non-noxious aversive stimuli either failed to activate dopamine neurons or, in cases of close resemblance with appetitive stimuli, induced weaker responses than appetitive stimuli. Thus, dopamine neurons preferentially report environmental stimuli with appetitive rather than aversive motivational value.
Animals↗
[Differential indications for so-called "lateral release" in treatment of chondropathia patellae].
The success rate of the operation of lateral release for pain caused by the patella is reported as being between 14% and 99%. The choice between arthroscopic or open procedures does not seem to affect the results. The wide ranges of results probably reflects differences in patients selection or the method and investigations of follow up. The early term outcome usually show better results than long term follow up. This study evaluates the indications for the operation of lateral releases and discusses the result of 36 out of a total of 42 patients who were follow up for 3 years later surgery. We found that an insufficiency of dysplasia of the vastus medialis, the laxity of the capsule and soft ligaments, and a strong lateral retinaculum were important factors in the indication for this procedure. The quadriceps angle was also of prime importance, but the minor forms of patellar dysplasia played only a minor role. In the so-called hyperpression syndrome, where the patella has a strong tendency to move laterally, the simple lateral release is the single most successful operation. The indication for procedures additionally to the lateral release is examined. We found that in a case with an insufficiently guided patella, a weak capsule and ligaments, an additional capsule roughing should be performed. The presence of early degenerative changes in the joint predisposes to poor results in operations such as abrasion and pride drilling. The results in our study were assessed using the Lysholm score. Our results show that the most successful technique was the combination of an arthroscopy and an extraarticular open operation controlled by arthroscopic means. This technique was not associated with major complications such as haemarthrosis and consecutive prolonged postoperative rehabilitation. Overall we achieved a rate of 83% of good or satisfactory results at more than 3 years using the indications and techniques described above.
Adult↗
Importance of unpredictability for reward responses in primate dopamine neurons.
1. We used single neuron recording techniques in two behaving monkeys to investigate the conditions in which dopamine neurons respond to primary rewarding or potentially rewarding stimuli. Animals received drops of liquid either outside behavioral tasks or as rewards during learning or established performance of an auditory reaction time task. 2. Three quarters of dopamine neurons showed a short-latency, phasic response to liquid that was delivered outside the task without being predicted by phasic stimuli. The same neurons responded to liquid reward during learning but not when task performance was established, at which time the neuronal response occurred to the conditioned, reward-predicting, movement-triggering stimulus. 3. These data suggest that the responses of dopamine neurons to rewarding or potentially rewarding liquid are due to the temporally unpredicted stimulus occurrence. A known, reward-predicting, tonic context does not prevent dopamine neurons from responding to the rewarding liquid. The responses during learning apparently occur because reward is not yet reliably predicted by a conditioned phasic stimulus. Because the unpredicted occurrence of reward is of central importance for learning, these responses allow dopamine neurons to play an important role in reward-driven learning.
Animals↗
Light and electron microscopical immunohistochemical localization of the small proteoglycan core proteins decorin and biglycan in human knee joint cartilage.
The distribution of decorin and biglycan was investigated at the light and electron microscopical level in adult human articular cartilage. In general, the amount of decorin and biglycan was found to decrease with the depth of the layer of the cartilage. Decorin was found in the interterritorial matrix where most of the collagen is located. This fits in well with the assumption that decorin may modulate collagen metabolism. Biglycan was found next to the chondrocytes in the pericellular matrix and is assumed to be responsible for cellular activities. At the ultrastructural level, decorin was localized in the interterritorial matrix and in vesicles in chondrocytes. Biglycan was found, usually though not exclusively in the pericellular matrix. Both small proteoglycans were detected close to and on the collagen fibres and also associated with the more globular structures of the matrix between the fibrils. A double-staining approach revealed that the two molecules could be located along the same collagen fibril. However, staining for biglycan and decorin was not observed simultaneously within the same region of the fibre.
Adult↗
The Langat model for tick-borne encephalitis virus. Specific detection by RT-PCR.
We have developed a reverse-transcriptase polymerase chain reaction assay for rapid detection of Langat (LGT) virus, a flavivirus that is closely related to the highly pathogenic tick-borne encephalitis (TBE) viruses. Unlike TBE viruses, LGT virus exhibits a significantly lower virulence for man. The assay serves as a safe alternative for the development and optimization of specific assays for the highly pathogenic subtypes of TBE viruses that are endemic throughout much of Europe, the former Soviet Union, and China.
Animals↗
Reward-related activity in the monkey striatum and substantia nigra.
Explore the source record for details and available documents.
Animals↗
Perceptual adaptation in the vestibulo-ocular system: EEG correlates of spatial and temporal rearrangement.
Spatial-temporal visuomotor rearrangement caused pursuit eye movements to counteract the vestibulo-ocular reflex (VOR). Vertical head nodding produced horizontal oscillations of a light spot delayed 1 or 150 msec. Adaptation resulted in apparent complementary motion of a stationary stimulus during nodding. 16 subjects adapted and tested at 150 msec. showed a 10% magnitude apparent motion. Following normal vision, while the electroencephalogram (EEG) was recorded, subjects were readapted at 150 msec. but tested at 1 msec. (to measure temporal generalization). Individual performance was correlated with EEG alpha. Adaptation correlated negatively with O(z) and Fz intensity, and positively with O(z) frequency. Temporal generalization correlated positively with O(z) intensity and negatively with O(z)-Fz phase angle. These results suggest that visuomotor adaptability is related to electrocortical activity.
Adult↗
Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task.
The present investigation had two aims: (1) to study responses of dopamine neurons to stimuli with attentional and motivational significance during several steps of learning a behavioral task, and (2) to study the activity of dopamine neurons during the performance of cognitive tasks known to be impaired after lesions of these neurons. Monkeys that had previously learned a simple reaction time task were trained to perform a spatial delayed response task via two intermediate tasks. During the learning of each new task, a total of 25% of 76 dopamine neurons showed phasic responses to the delivery of primary liquid reward, whereas only 9% of 163 neurons responded to this event once task performance was established. This produced an average population response during but not after learning of each task. Reward responses during learning were significantly more numerous and pronounced in area A10, as compared to areas A8 and A9. Dopamine neurons also showed phasic responses to the two conditioned stimuli. These were the instruction cue, which was the first stimulus in each trial and indicated the target of the upcoming arm movement (58% of 76 neurons during and 44% of 163 neurons after learning), and the trigger stimulus, which was a conditioned incentive stimulus predicting reward and eliciting a saccadic eye movement and an arm reaching movement (38% of neurons during and 40% after learning). None of the dopamine neurons showed sustained activity in the delay between the instruction and trigger stimuli that would resemble the activity of neurons in dopamine terminal areas, such as the striatum and frontal cortex. Thus, dopamine neurons respond phasically to alerting external stimuli with behavioral significance whose detection is crucial for learning and performing delayed response tasks. The lack of sustained activity suggests that dopamine neurons do not encode representational processes, such as working memory, expectation of external stimuli or reward, or preparation of movement. Rather, dopamine neurons are involved with transient changes of impulse activity in basic attentional and motivational processes underlying learning and cognitive behavior.
Animals↗