PubMed Health⌕ Search

SEARCH · PubMed Health

Results for “Reinforcement learning”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56Linked to original sources
PubMed1983

Endogenous opioids: naloxone disrupts learned performance in rats.

The effect of naloxone on learned performance reinforced by food was examined in 2 experiments. Male rats were trained to run down a short runway for 5 (45 mg) food pellets per trial and were then shifted either to 1 or 0 pellets. Following such an abrupt reinforcement shift, animals typically show an emotional disruption of performance (Crespi, 1942) referred to as "depression". We examined the postshift depression-effect in groups treated either with saline (SAL) or naloxone (NAL). In experiment 1 NAL groups received a single 10 mg/kg (s.c.) injection prior to each postshift session. When compared with SAL controls, NAL animals showed an exaggerated postshift depression-effect. Furthermore, a single (0.3 mg/kg, s.c.) injection of the enkephalin analog FK 33-824 (D-Ala2, MePhe4, Met-(0)5-o1) produced a dramatic recovery of performance. In the second experiment, these effects were replicated at a low NAL dose (1 mg/kg), which had no direct effect on motor performance. These findings suggest that opiate systems may modulate the incentive motivation that maintains learned performance.

Animals↗
PubMed1980

On the discriminability of stimulus duration.

The performance of pigeons trained to detect differences in the duration of stimuli was analysed using a matching model of signal detection. Two white stimuli, S1 and S2, differing in duration, were arranged with equal probability on the center key of a three-key chamber. S1 was systematically varied from 5 seconds to 25 seconds while S2 remained constant at 30 seconds. On completion of the center-key stimulus, a peck on the center key turned on the two red side keys. A left-key response was "correct" when S1 had been in effect on the center key and a right-key response was "correct" on S2 trials. A correct response produced a 3-second magazine light accompanied intermittently by food. Incorrect responses produced 3-second blackouts. Detection performance was measured under two procedures. In the first, the obtained reinforcement ratio was uncontrolled by allowing the number of food reinforcements obtained for correct left- and right-key responses to vary as the stimuli were changed. In the second procedure, the presentation of food reinforcement was controlled by holding the obtained reinforcement ratio constant. Discriminability changed as a function of stimulus differences under both procedures. No such trend was found in response bias.

Animals↗
PubMed2003

Every reinforcer counts: reinforcer magnitude and local preference.

Six pigeons were trained on concurrent variable-interval schedules. Sessions consisted of seven components, each lasting 10 reinforcers, with the conditions of reinforcement differing between components. The component sequence was randomly selected without replacement. In Experiment 1, the concurrent-schedule reinforcer ratios in components were all equal to 1.0, but across components reinforcer-magnitude ratios varied from 1:7 through 7:1. Three different overall reinforcer rates were arranged across conditions. In Experiment 2, the reinforcer-rate ratios varied across components from 27:1 to 1:27, and the reinforcer-magnitude ratios for each alternative were changed across conditions from 1:7 to 7:1. The results of Experiment 1 replicated the results for changing reinforcer-rate ratios across components reported by Davison and Baum (2000, 2002): Sensitivity to reinforcer-magnitude ratios increased with increasing numbers of reinforcers in components. Sensitivity to magnitude ratio, however, fell short of sensitivity to reinforcer-rate ratio. The degree of carryover from component to component depended on the reinforcer rate. Larger reinforcers produced larger and longer postreinforcer preference pulses than did smaller reinforcers. Similar results were found in Experiment 2, except that sensitivity to reinforcer magnitude was considerably higher and was greater for magnitudes that differed more from one another. Visit durations following reinforcers measured either as number of responses emitted or time spent responding before a changeover were longer following larger than following smaller reinforcers, and were longer following sequences of same reinforcers than following other sequences. The results add to the growing body of research that informs model building at local levels.

Animals↗
PubMed1964

ATTITUDE STATEMENTS AS POSITIVE AND NEGATIVE REINFORCEMENTS.

The finding that attraction is a function of attitude similarity has been interpreted as a special case of the effect of positive and negative reinforcements on attraction. A simple discrimination learning task was employed in which the reinforcements were attitude statements similar and dissimilar to the opinions of the subject. The presentation of similar attitude statements after each correct response and dissimilar attitude statements after each incorrect response significantly changed response probability. The hypothesis that such statements could be used as reinforcers in a learning situation was thus confirmed.

Attitude↗
PubMed1979

Signal probability, reinforcement and signal detection.

Five pigeons were trained to detect differences in light intensity. Two stimuli, S1 and S2, differing in intensity, were arranged on the center key of a three-key chamber according to set probabilities. A peck on the center key turned on the two side keys. When S1 was presented on the center key, a peck on the left key was "correct" and when S2 was presented, a peck on the right key was "correct." Correct responses produced reinforcement and incorrect responses produced 3-second blackout. Detection performance was measured under three procedures. The first was a standard signal-detection design in which the probability of S1 was varied and the number of reinforcements obtained for correct responses to S1 was allowed to covary. In the second procedure, the probability of S1 was again varied but the distribution of reinforcements between the two choices was kept equal. In the third procedure, probability of S1 was held constant while the distribution of reinforcements was varied between the two choices. Changes in response bias were a function of variations in the relative reinforcement ratio for the choice responses and not a function of variations in the probability of stimulus presentation. Discriminability remained constant across the three procedures.

Animals↗
PubMed1995

Instrumental learning in toads (Bufo arenarum): reinforcer magnitude and the medial pallium.

Two experiments studied the effects of shifts in the amount of water reinforcement on the instrumental performance of toads (Bufo arenarum) using one trial per day. In the first experiment, a shift from a large to a small reward magnitude led to a gradual change in performance without evidence of negative contrast. A shift from large or small reward magnitude to extinction led to similar extinction rates and provided no indication of the magnitude of reinforcement extinction effect. In the second experiment, the gradual change in performance (with no contrast) after a shift from a large to a small reward magnitude was completely eliminated by ablation of the medial pallium. These results provide support for the hypothesis that the medial pallium in amphibians is homologous to the hippocampal formation in mammals and also indicate that species differences in these learning phenomena may be related to the differential development and differentiation of the hippocampal formation.

Animals↗
PubMed2004

Learning in neural networks by reinforcement of irregular spiking.

Artificial neural networks are often trained by using the back propagation algorithm to compute the gradient of an objective function with respect to the synaptic strengths. For a biological neural network, such a gradient computation would be difficult to implement, because of the complex dynamics of intrinsic and synaptic conductances in neurons. Here we show that irregular spiking similar to that observed in biological neurons could be used as the basis for a learning rule that calculates a stochastic approximation to the gradient. The learning rule is derived based on a special class of model networks in which neurons fire spike trains with Poisson statistics. The learning is compatible with forms of synaptic dynamics such as short-term facilitation and depression. By correlating the fluctuations in irregular spiking with a reward signal, the learning rule performs stochastic gradient ascent on the expected reward. It is applied to two examples, learning the XOR computation and learning direction selectivity using depressing synapses. We also show in simulation that the learning rule is applicable to a network of noisy integrate-and-fire neurons.

Action Potentials↗
PubMed2006

Effects of partial reinforcement and time between reinforced trials on terminal response rate in pigeon autoshaping.

Partial reinforcement often leads to asymptotically higher rates of responding and number of trials with a response than does continuous reinforcement in pigeon autoshaping. However, comparisons typically involve a partial reinforcement schedule that differs from the continuous reinforcement schedule in both time between reinforced trials and probability of reinforcement. Two experiments examined the relative contributions of these two manipulations to asymptotic response rate. Results suggest that the greater responding previously seen with partial reinforcement is primarily due to differential probability of reinforcement and not differential time between reinforced trials. Further, once established, differences in responding are resistant to a change in stimulus and contingency. Secondary response theories of autoshaped responding (theories that posit additional response-augmenting or response-attenuating mechanisms specific to partial or continuous reinforcement) cannot fully accommodate the current body of data. It is suggested that researchers who study pigeon autoshaping train animals on a common task prior to training them under different conditions.

Animals↗
PubMed1962

Temporal discrimination in pigeons.

Pigeons trained to peck a lighted key were presented with a key that was alternately dark and lighted. The key was dark for intervals of from 3 to 30 seconds. Pecking of the lighted key was reinforced only after the shortest or, in a second experiment, the longest interval that the key was dark. The pigeons were able to discriminate the duration of the dark interval.

Animals↗
PubMed1992

Behavioral contrast: Pavlovian effects and anticipatory contrast.

Two sources of behavioral contrast have been identified previously: Pavlovian stimulus-reinforcer relations and component sequence effects (anticipatory contrast). This study sought to isolate these sources of control procedurally in a four-ply multiple schedule composed of two fixed two-component sequences. Different cues were associated with the first component of each sequence, and contrast effects were studied in these target components. In Experiment 1, differential cuing of Component 2 between sequences and availability of reinforcement during target components were varied across three groups of pigeons; the stimulus-reinforcer relation between target-component cues and schedule of reinforcement in Component 2 was varied within subjects. Control by the Pavlovian relation was demonstrated under all conditions, and anticipatory contrast was not observed. In Experiment 2, target-component duration was systematically varied in the three groups of Experiment 1. Control by the Pavlovian relation was reliably obtained only when target-component behavior was unreinforced, and diminished with increases in component duration. Anticipatory contrast emerged in the two groups for which target-component reinforcement was available. These and other data indicate that Pavlovian effects in multiple schedules may be obscured when the requisite conditions for anticipatory contrast are present.

Animals↗
PubMed1979

Concurrent schedules: undermatching and control by previous experimental conditions.

Five pigeons were trained on concurrent variable-interval schedules. A series of conditions in which the ratio of reinforcement rates on two keys was progressively increased and then decreased was arranged twice. The birds were then exposed to an irregular sequence of conditions. Each condition in which reinforcement was available on both keys lasted six sessions. Performance in the first, third, and sixth sessions after a condition change was analyzed. Following a condition change, preference was biased toward the preference in the last condition, but this effect largely disappeared before the sixth session of training. The birds' preferences also appeared less sensitive to reinforcement rates in early sessions after a transition. Preference in a session was a function of both the reinforcements in that session and the reinforcements obtained in as many as four or five previous sessions. The effects of reinforcements in previous sessions could be summarized by the performance in the immediately preceding session, giving a relatively simple relation between present performance and a combination of present reinforcement and prior session performance. While such hysteresis could cause undermatching when only a small number of sessions are arranged in a condition, undermatching in a stable-state performance probably arises elsewhere.

Animals↗
PubMed1981

Learning improvement of appetitively or aversively reinforced light-dark discrimination and reversal four weeks after electrical stimulation of the lateral hypothalamus of the rat.

The experiments reported here point out the effect of electrical stimulation of the lateral hypothalamus on the acquisition and reversal of a simultaneous light-dark discrimination performed 4 weeks later. The stimulation was applied on 15-day-old rats. There were two control groups, implanted but not stimulated and non-implanted rats. In the first experiment, the light-dark discrimination task was food-reinforced. As already reported in the case of an appetitively reinforced operant task, stimulation of the lateral hypothalamus induced an important improvement of performance in discrimination and reversal. This improvement was already apparent at the beginning of acquisition. In the reversal task the mean response latency of stimulated rats was greater than tht of control animals. In the second experiment with naive rats we used the same discrimination but the task was aversively reinforced by an electric foot-shock. In spite of the different type of reinforcement used, the stimulated rats performed better than the control rats in the three behavioral situations studied: avoidance, discrimination and reversal. As in the first experiment the improvement appeared at the onset of acquisition in the three situations. Thus the long-term effect of lateral hypothalamic stimulation does not depend on the type of reinforcement, nor the behavioral task. These results are briefly discussed.

Animals↗
PubMedDate not supplied

[The effect of serotonin on the cross-correlation of neuronal activity in surviving cortical slices].

Serotonin-induced changes in crosscorrelation functions of the trains of neuronal activity were studied in surviving slices of the guinea pig cerebral cortex. After serotonin administration into the incubation medium and its washing-out the value of the crosscorrelation function in the zero bin increased as compared with its initial value if the background impulsation rate of both neurons was high and decreased if the background rate was low. This finding substantiates the earlier advanced hypothesis that serotoninergic system is related to the positive reinforcement in learning and influences the efficiency of the cortico-cortical synapses increasing it when the initial excitation of the presynaptic neuron is high and decreasing when the background presynaptic excitation level is low.

Animals↗
PubMed1990

Suboptimal choice in a percentage-reinforcement procedure: effects of signal condition and terminal-link length.

Pigeons' choice between reliable (100%) and unreliable (50%) reinforcement was studied using a concurrent-chains procedure. Initial links were fixed-ratio 1 schedules, and terminal links were equal fixed-time schedules. The duration of the terminal links was varied across conditions. The terminal link on the reliable side always ended in food; the terminal link on the unreliable side ended with food 50% of the time and otherwise with blackout. Different stimuli present during the 50% terminal links signaled food or blackout outcomes under signaled conditions but were uncorrelated with outcomes under unsignaled conditions. In signaled conditions, most pigeons displayed a nearly exclusive preference for the 100% alternative when terminal links were short (5 or 10 s), but with terminal links of 30 s or longer, preference for the 100% alternative was sharply reduced (often to below .5). In unsignaled conditions, most pigeons showed extreme preference for the 100% alternative with either short (5 s) or longer (30 s) terminal links. Thus, pigeons' choice between reliable and unreliable reinforcement is influenced by both the signal conditions on the unreliable alternative and the duration of the terminal-link delay. With a long delay and signaled outcomes, many pigeons display a suboptimal tendency to choose the unreliable side.

Animals↗
PubMed2006

The influence of prior choices on current choice.

Three pigeons chose between random-interval (RI) and tandem, continuous-reinforcement, fixed-interval (crf-FI) reinforcement schedules by pecking either of two keys. As long as a pigeon pecked on the RI key, both keys remained available. If a pigeon pecked on the crf-FI key, then the RI key became unavailable and the crf-FI timer began to time out. With this procedure, once the RI key was initially pecked, the prospective value of both alternatives remained constant regardless of time spent pecking on the RI key without reinforcement (RI waiting time). Despite this constancy, the rate at which pigeons switched from the RI to the crf-FI decreased sharply as RI waiting time increased. That is, prior choices influenced current choice-an exercise effect. It is argued that such influence (independent of reinforcement contingencies) may serve as a sunk-cost commitment device in self-control situations. In a second experiment, extinction was programmed if RI waiting time exceeded a certain value. Rate of switching to the crf-FI first decreased and then increased as the extinction point approached, showing sensitivity to both prior choices and reinforcement contingencies. In a third experiment, crf-FI availability was limited to a brief window during the RI waiting time. When constrained in this way, switching occurred at a high rate regardless of when, during the RI waiting time, the crf-FI became available.

Animals↗