Biomedical subjects

Rudolf N Cardinal

Publications and source records attributed to Rudolf N Cardinal.

15 recordsLinked to original sources

PubMed2006

Neural systems implicated in delayed and probabilistic reinforcement.

This review considers the theoretical problems facing agents that must learn and choose on the basis of reward or reinforcement that is uncertain or delayed, in implicit or procedural (stimulus-response) representational systems and in explicit or declarative (action-outcome-value) representational systems. Individual differences in sensitivity to delays and uncertainty may contribute to impulsivity and risk taking. Learning and choice with delayed and uncertain reinforcement are related but in some cases dissociable processes. The contributions to delay and uncertainty discounting of neuromodulators including serotonin, dopamine, and noradrenaline, and of specific neural structures including the nucleus accumbens core, nucleus accumbens shell, orbitofrontal cortex, basolateral amygdala, anterior cingulate cortex, medial prefrontal (prelimbic/infralimbic) cortex, insula, subthalamic nucleus, and hippocampus are examined.

Animals↗

PubMed2005

Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats.

BACKGROUND: Animals must frequently make choices between alternative courses of action, seeking to maximize the benefit obtained. They must therefore evaluate the magnitude and the likelihood of the available outcomes. Little is known of the neural basis of this process, or what might predispose individuals to be overly conservative or to take risks excessively (avoiding or preferring uncertainty, respectively). The nucleus accumbens core (AcbC) is known to contribute to rats' ability to choose large, delayed rewards over small, immediate rewards; AcbC lesions cause impulsive choice and an impairment in learning with delayed reinforcement. However, it is not known how the AcbC contributes to choice involving probabilistic reinforcement, such as between a large, uncertain reward and a small, certain reward. We examined the effects of excitotoxic lesions of the AcbC on probabilistic choice in rats. RESULTS: Rats chose between a single food pellet delivered with certainty (p = 1) and four food pellets delivered with varying degrees of uncertainty (p = 1, 0.5, 0.25, 0.125, and 0.0625) in a discrete-trial task, with the large-reinforcer probability decreasing or increasing across the session. Subjects were trained on this task and then received excitotoxic or sham lesions of the AcbC before being retested. After a transient period during which AcbC-lesioned rats exhibited relative indifference between the two alternatives compared to controls, AcbC-lesioned rats came to exhibit risk-averse choice, choosing the large reinforcer less often than controls when it was uncertain, to the extent that they obtained less food as a result. Rats behaved as if indifferent between a single certain pellet and four pellets at p = 0.32 (sham-operated) or at p = 0.70 (AcbC-lesioned) by the end of testing. When the probabilities did not vary across the session, AcbC-lesioned rats and controls strongly preferred the large reinforcer when it was certain, and strongly preferred the small reinforcer when the large reinforcer was very unlikely (p = 0.0625), with no differences between AcbC-lesioned and sham-operated groups. CONCLUSION: These results support the view that the AcbC contributes to action selection by promoting the choice of uncertain, as well as delayed, reinforcement.

Animals↗

PubMed2005

Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats.

BACKGROUND: Animals must frequently act to influence the world even when the reinforcing outcomes of their actions are delayed. Learning with action-outcome delays is a complex problem, and little is known of the neural mechanisms that bridge such delays. When outcomes are delayed, they may be attributed to (or associated with) the action that caused them, or mistakenly attributed to other stimuli, such as the environmental context. Consequently, animals that are poor at forming context-outcome associations might learn action-outcome associations better with delayed reinforcement than normal animals. The hippocampus contributes to the representation of environmental context, being required for aspects of contextual conditioning. We therefore hypothesized that animals with hippocampal lesions would be better than normal animals at learning to act on the basis of delayed reinforcement. We tested the ability of hippocampal-lesioned rats to learn a free-operant instrumental response using delayed reinforcement, and what is potentially a related ability -- the ability to exhibit self-controlled choice, or to sacrifice an immediate, small reward in order to obtain a delayed but larger reward. RESULTS: Rats with sham or excitotoxic hippocampal lesions acquired an instrumental response with different delays (0, 10, or 20 s) between the response and reinforcer delivery. These delays retarded learning in normal rats. Hippocampal-lesioned rats responded slightly less than sham-operated controls in the absence of delays, but they became better at learning (relative to shams) as the delays increased; delays impaired learning less in hippocampal-lesioned rats than in shams. In contrast, lesioned rats exhibited impulsive choice, preferring an immediate, small reward to a delayed, larger reward, even though they preferred the large reward when it was not delayed. CONCLUSION: These results support the view that the hippocampus hinders action-outcome learning with delayed outcomes, perhaps because it promotes the formation of context-outcome associations instead. However, although lesioned rats were better at learning with delayed reinforcement, they were worse at choosing it, suggesting that self-controlled choice and learning with delayed reinforcement tax different psychological processes.

Animals↗

PubMed2005

Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex during a test of impulsive choice.

Dysregulation of the prefrontal cortex (PFC) has been implicated in impulse control disorders, including attention deficit hyperactivity disorder. A growing body of evidence suggests that impulsivity is non-unitary in nature, and recent data indicate that the ventral and dorsal regions of the PFC are differentially involved in distinct aspects of impulsive behaviour, findings which may reflect differences in the monoaminergic regulation of these regions. In the current experiment, levels of dopamine, serotonin and their metabolites were measured in the medial PFC (n = 12) and orbitofrontal cortex (OFC) (n = 19) of rats using in vivo microdialysis during the delay-discounting model of impulsive choice, where impulsivity is defined as selection of small immediate over larger delayed rewards. Yoked groups were also dialysed to control for instrumental responding and reward delivery. Significant increases in 5-hydroxytryptamine efflux were observed in the mPFC, but not in the OFC, during task performance but not under yoked control conditions. In the OFC, 3,4-di-hydroxy-phenylocetic acid (DOPAC) levels increased in animals performing the task but not in yoked animals, whereas mPFC DOPAC levels increased in all subjects. These data suggest a double dissociation between serotonergic and dopaminergic modulation of impulsive decision-making within distinct areas of frontal cortex.

Animals↗

PubMed2005

Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat.

BACKGROUND: Delays between actions and their outcomes severely hinder reinforcement learning systems, but little is known of the neural mechanism by which animals overcome this problem and bridge such delays. The nucleus accumbens core (AcbC), part of the ventral striatum, is required for normal preference for a large, delayed reward over a small, immediate reward (self-controlled choice) in rats, but the reason for this is unclear. We investigated the role of the AcbC in learning a free-operant instrumental response using delayed reinforcement, performance of a previously-learned response for delayed reinforcement, and assessment of the relative magnitudes of two different rewards. RESULTS: Groups of rats with excitotoxic or sham lesions of the AcbC acquired an instrumental response with different delays (0, 10, or 20 s) between the lever-press response and reinforcer delivery. A second (inactive) lever was also present, but responding on it was never reinforced. As expected, the delays retarded learning in normal rats. AcbC lesions did not hinder learning in the absence of delays, but AcbC-lesioned rats were impaired in learning when there was a delay, relative to sham-operated controls. All groups eventually acquired the response and discriminated the active lever from the inactive lever to some degree. Rats were subsequently trained to discriminate reinforcers of different magnitudes. AcbC-lesioned rats were more sensitive to differences in reinforcer magnitude than sham-operated controls, suggesting that the deficit in self-controlled choice previously observed in such rats was a consequence of reduced preference for delayed rewards relative to immediate rewards, not of reduced preference for large rewards relative to small rewards. AcbC lesions also impaired the performance of a previously-learned instrumental response in a delay-dependent fashion. CONCLUSIONS: These results demonstrate that the AcbC contributes to instrumental learning and performance by bridging delays between subjects' actions and the ensuing outcomes that reinforce behaviour.

Animals↗

PubMed2004

Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice.

The orbitofrontal cortex (OFC) and basolateral nucleus of the amygdala (BLA) share many reciprocal connections, and a functional interaction between these regions is important in controlling goal-directed behavior. However, their relative roles have proved hard to dissociate. Although injury to these brain regions can cause similar effects, it has been suggested that the resulting impairments arise through damage to different, yet converging, cognitive processes. Patients with OFC or amygdala lesions exhibit maladaptive decision making and aberrant social behavior often described as impulsive. Impulsive choice may be measured in both humans and rodents by evaluating intolerance to delay of reinforcement. Rats with excitotoxic lesions of the BLA and OFC were tested on such a delay-discounting procedure. Although lesions of the BLA increased choice of the small immediate reward, indicating greater impulsivity, OFC lesions had the opposite effect, increasing preference for the larger but delayed reward. The fact that the delay did not devalue the large reward to such an extent in OFC-lesioned animals supports the suggestion that the OFC is involved in updating the incentive value of outcomes in response to devaluation. In contrast, the BLA-lesioned animals markedly decreased their preference for the large reward when it was delayed, potentially because of an inability to maintain a representation of the reward in its absence. This is the first time that lesions to these two structures have produced opposite behavioral effects, indicating their distinct contributions to cognition.

Amygdala↗

PubMed2004

Cortical cholinergic function and deficits in visual attentional performance in rats following 192 IgG-saporin-induced lesions of the medial prefrontal cortex.

Lesions of the basal forebrain (BF) cortical cholinergic system impair performance on a rodent five-choice visual attentional task. This study examines the effects on the same task of selective depletion of acetylcholine from the prefrontal cortex (PFC) using 192 IgG-saporin, the cholinergic immunotoxin. Rats were trained to detect brief visual stimuli, either presented unpredictably both temporally and spatially to increase attentional load, or under less demanding conditions where stimuli were temporally and spatially predictable. Following training, 192 IgG-saporin (50 ng or 100 ng/infusion) or its vehicle was infused bilaterally into the ventromedial PFC. The 100 ng lesion group exhibited post-operatively a transient increase in perseveration, specifically when the visual stimuli were temporally unpredictable. A vigilance decrement, as well as a reinstatement of perseverative responding occurred in both lesion groups under conditions of enhanced attentional load, specifically with high target frequency sustained over many trials. Lesioned subjects were also more impulsive with increased anticipatory errors. Systemic administration of the muscarinic receptor antagonist scopolamine further dissociated the groups with attentional accuracy in the 100 ng group decreasing relative to shams. These findings are consistent with an important modulatory influence of PFC function by BF cholinergic neurons, particularly during increased attentional demand.

Acetylcholine↗

PubMed2004

Neural and psychological mechanisms underlying appetitive learning: links to drug addiction.

The complexity of drug addiction mirrors the complexity of the psychological processes that motivate animals to work for any reinforcer, be it a natural reward or a drug. Here, we review the role of the nucleus accumbens, together with its dopaminergic and cortical innervation, in responding to reinforcement. One important contribution made by the nucleus accumbens is to the process through which neutral stimuli, once paired with a reinforcer such as a drug, have the capacity to motivate behaviour. This process may be one of several contributing to addiction, and it may be amenable to pharmacological intervention.

Animals↗

PubMed2004

Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates.

The prefrontal cortex has been implicated in a variety of cognitive and executive processes, including working memory, decision-making, inhibitory response control, attentional set-shifting and the temporal integration of voluntary behaviour. This article reviews current progress in our understanding of the rodent prefrontal cortex, especially evidence for functional divergence of the anatomically distinct sub-regions of the rat prefrontal cortex. Recent findings suggest clear distinctions between the dorsal (precentral and anterior cingulate) and ventral (prelimbic, infralimbic and medial orbital) sub-divisions of the medial prefrontal cortex, and between the orbitofrontal cortex (ventral orbital, ventrolateral orbital, dorsal and ventral agranular cortices) and the adjacent medial wall of the prefrontal cortex. The dorso-medial prefrontal cortex is implicated in memory for motor responses, including response selection, and the temporal processing of information. Ventral regions of the medial prefrontal cortex are implicated in interrelated 'supervisory' attentional functions, including attention to stimulus features and task contingencies (or action-outcome rules), attentional set-shifting, and behavioural flexibility. The orbitofrontal cortex is implicated in lower-order discriminations, including reversal of stimulus-reward associations (reversal learning), and choice involving delayed reinforcement. It is anticipated that a greater understanding of the prefrontal cortex will come from using tasks that load specific cognitive and executive processes, in parallel with discovering new ways of manipulating the different sub-regions and neuromodulatory systems of the prefrontal cortex.

Acetylcholine↗

PubMed2004

Limbic corticostriatal systems and delayed reinforcement.

Impulsive choice, one aspect of impulsivity, is characterized by an abnormally high preference for small, immediate rewards over larger delayed rewards, and can be a feature of adolescence, but also attention-deficit/hyperactivity disorder (ADHD), addiction, and other neuropsychiatric disorders. Both the serotonin and dopamine neuromodulator systems are implicated in impulsivity; manipulations of these systems affect animal models of impulsive choice, though these effects may depend on the receptor subtype and whether or not the reward is signaled. These systems project to limbic cortical and striatal structures shown to be abnormal in animal models of ADHD. Damage to the nucleus accumbens core (AcbC) causes rats to exhibit impulsive choice. These rats are also hyperactive, but are unimpaired in tests of visuospatial attention; they may therefore represent an animal model of the hyperactive-impulsive subtype of ADHD. Lesions to the anterior cingulate or medial prefrontal cortex, two afferents to the AcbC, do not induce impulsive choice, but lesions of the basolateral amygdala do, while lesions to the orbitofrontal cortex have had opposite effects in different tasks measuring impulsive choice. In theory, impulsive choice may emerge as a result of abnormal processing of the magnitude of rewards, or as a result of a deficit in the effects of delayed reinforcement. Recent evidence suggests that AcbC-lesioned rats perceive reward magnitude normally, but exhibit a selective deficit in learning instrumental responses using delayed reinforcement, suggesting that the AcbC is a reinforcement learning system that mediates the effects of delayed rewards.

Adolescent↗

PubMed2003

Role of the anterior cingulate cortex in the control over behavior by Pavlovian conditioned stimuli in rats.

To investigate the contribution of the anterior cingulate cortex (ACC) to stimulus-reward learning, rats with lesions of peri- and postgenual ACC were tested on a variety of Pavlovian conditioning tasks. Lesioned rats learned to approach a food alcove during a stimulus predicting food, and responded normally for conditioned reinforcement. They also exhibited normal conditioned freezing and Pavlovian-instrumental transfer, yet were impaired at autoshaping. To resolve this apparent discrepancy, a further task was developed in which approach to the food alcove was under the control of 2 stimuli, only 1 of which was followed by reward. Lesioned rats were impaired, approaching during both stimuli. It is suggested that the ACC is not critical for stimulus-reward learning per se, but is required to discriminate multiple stimuli on the basis of their association with reward.

Amphetamine↗

PubMed2003

Appetitive behavior: impact of amygdala-dependent mechanisms of emotional learning.

In this chapter, we review data from studies involving appetitive conditioning using measures of pavlovian approach behavior and the effects of pavlovian conditioned stimuli on instrumental behavior, including the pavlovian-to-instrumental transfer effect and conditioned reinforcement. These studies consistently demonstrate double dissociations of function between the basolateral area and the central nucleus of the amygdala. Moreover, the data show marked parallels with data derived from aversive (fear) conditioning studies and are consistent with the idea that these subsystems of the amygdala mediate different kinds of associative representation formed during pavlovian conditioning. We hypothesize that the basolateral amygdala is required for a conditioned stimulus to gain access to the current affective value of its specific unconditioned stimulus, whereas the central nucleus mediates stimulus-response representations and conditioned motivational influences on behavior. Although these systems normally operate together, they can also modulate behavior in distinct ways. In many circumstances, then, emotional behavior can be seen as a coordinated combination of processing by these amygdaloid sub-nuclei, reflecting the superimposition of a phylogenetically recent basolateral amygdala subsystem that encodes and retrieves the affective value of environmental stimuli and thereby directs complex, adaptive behavioral responses onto a phylogenetically older central amygdala subsystem that enables cortical structures (including the basolateral amygdala) to recruit incentive motivational processes and thereby invigorate emotional responding.

Amygdala↗

PubMed2002

Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex.

Emotions are multifaceted, but a key aspect of emotion involves the assessment of the value of environmental stimuli. This article reviews the many psychological representations, including representations of stimulus value, which are formed in the brain during Pavlovian and instrumental conditioning tasks. These representations may be related directly to the functions of cortical and subcortical neural structures. The basolateral amygdala (BLA) appears to be required for a Pavlovian conditioned stimulus (CS) to gain access to the current value of the specific unconditioned stimulus (US) that it predicts, while the central nucleus of the amygdala acts as a controller of brainstem arousal and response systems, and subserves some forms of stimulus-response Pavlovian conditioning. The nucleus accumbens, which appears not to be required for knowledge of the contingency between instrumental actions and their outcomes, nevertheless influences instrumental behaviour strongly by allowing Pavlovian CSs to affect the level of instrumental responding (Pavlovian-instrumental transfer), and is required for the normal ability of animals to choose rewards that are delayed. The prelimbic cortex is required for the detection of instrumental action-outcome contingencies, while insular cortex may allow rats to retrieve the values of specific foods via their sensory properties. The orbitofrontal cortex, like the BLA, may represent aspects of reinforcer value that govern instrumental choice behaviour. Finally, the anterior cingulate cortex, implicated in human disorders of emotion and attention, may have multiple roles in responding to the emotional significance of stimuli and to errors in performance, preventing responding to inappropriate stimuli.

Amygdala↗

PubMed2002

Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats.

The nucleus accumbens core (AcbC), anterior cingulate cortex (ACC), and central nucleus of the amygdala (CeA) are required for normal acquisition of tasks based on stimulus-reward associations. However, it is not known whether they are involved purely in the learning process or are required for behavioral expression of a learned response. Rats were trained preoperatively on a Pavlovian autoshaping task in which pairing a visual conditioned stimulus (CS+) with food causes subjects to approach the CS+ while not approaching an unpaired stimulus (CS-). Subjects then received lesions of the AcbC, ACC, or CeA before being retested. AcbC lesions severely impaired performance; lesioned subjects approached the CS+ significantly less often than controls, failing to discriminate between the CS+ and CS-. ACC lesions also impaired performance but did not abolish discrimination entirely. CeA lesions had no effect on performance. Thus, the CeA is required for learning, but not expression, of a conditioned approach response, implying that it makes a specific contribution to the learning of stimulus-reward associations.

Amygdala↗

PubMedDate not supplied

Local analysis of behaviour in the adjusting-delay task for assessing choice of delayed reinforcement.

The adjusting-delay task introduced by Mazur (Quantitative analyses of behavior: V. The effect of delay and of intervening events on reinforcement value, 1987, pp. 55-73) has been widely used to study choice of delayed reinforcers. This paradigm involves repeated choice between one reinforcer delivered after a fixed delay and another, typically larger, reinforcer delivered after a variable delay; the variable delay is adjusted depending on the subject's choice until an equilibrium point is reached at which the subject is indifferent between the two alternatives. Rats were trained on a version of this task and their behaviour was examined to determine the nature of their sensitivity to the adjusting delay; these analyses included the use of a cross-correlational technique. No clear evidence of sensitivity to the adjusting delay was found. A number of decision rules, some sensitive to the adjusting delay and some not, were simulated and it was observed that some effects usually supposed to be a consequence of delay sensitivity could be generated by delay-independent processes, such as a consistent, unchanging relative preference between the alternatives. Consequently, the use of explicit analysis of delay sensitivity is advocated in future research on delayed reinforcement.

Animals↗