SEARCH · PubMed Health

Results for “representation learning”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMed2003

Associative learning and representation. An EPS workshop for N. J. Mackintosh, 9 July 2002.

The papers published in this Special Issue are based upon presentations at a workshop on Associative Learning and Representation, which was sponsored by the Experimental Psychology Society and hosted by Emmanuel College, Cambridge. The workshop celebrated the contribution of Professor Nicholas Mackintosh to animal learning and conditioning in particular and experimental psychology in general in the year of his retirement from the Chair of Psychology at the University of Cambridge after 21 years in post. The date of the workshop, 9 July 2002, was particularly auspicious as it was the day of Professor Mackintoshs birth 67 years ago. Moreover, it is particularly fitting that this tribute is published in the Comparative and Physiological Psychology Section (B) of Quarterly Journal of Experimental Psychology, which he founded in 1981 during his editorship of the Journal between 1977 and 1984.

Animals↗

PubMed1989

Hebbian learning reconsidered: representation of static and dynamic objects in associative neural nets.

According to Hebb's postulate for learning, information presented to a neural net during a learning session is stored in synaptic efficacies. Long-term potentiation occurs only if the postsynaptic neuron becomes active in a time window set up by the presynaptic one. We carefully interpret and mathematically implement the Hebb rule so as to handle both stationary and dynamic objects such as single patterns and cycles. Since the natural dynamics contains a rather broad distribution of delays, the key idea is to incorporate these delays in the learning session. As theory and numerical simulations show, the resulting procedure is surprisingly robust and faithful. It also turns out the pure Hebbian learning is by selection: the network produces synaptic representations that are selected according to their resonance with the input percepts.

Humans↗

PubMed1990

Three-dimensional computergraphic analysis of apraxia. Neural representations of learned movement.

The left cerebral hemisphere in man contains anatomical structures specialized not only for language but also for higher-order motor programming. One method of studying the nature of these motor programs is by observing the type of errors made by patients who have left hemisphere damage. A major problem, however, in investigating the disorders that result from failure of this specialized left hemisphere system (the apraxias) has been the difficulty in obtaining objective measurement of movement in three-dimensional space. To this end, we provide the first three-dimensional analysis of the nature of movement errors in apraxia. Two apraxic subjects with lesions to the left hemisphere and 5 matched control subjects were studied. The apraxic subjects showed impairments in the control of movement timing and spatial relations, as well as decoupling in the normally tight relation between certain spatial and temporal aspects of their movement trajectories. Further, the use of the distal musculature by apraxic subjects was more impaired than their use of the proximal musculature, suggesting more distal representation in any space-time maps of learned movement. These data provide further insight into the nature of the representations of learned skilled movements in the left cerebral hemisphere.

Aged↗

PubMed2005

Quality of phonological representations, verbal learning, and phoneme awareness in dyslexic and normal readers.

This study of dyslexia was concerned with the quality of phonological representations of lexical items. It extended the studies of verbal learning in dyslexia from learning new vocabulary items (pseudo-names) to the learning of more well-specified variants of known words. The participants were 19 dyslexic adolescents in grades 4 to 6 and 19 younger normal readers in grade 2 matched on single word decoding. The dyslexics were significantly outperformed by the reading-age controls in non-word reading and in phoneme awareness. The dyslexics also took longer time to learn to associate a set of pseudo-names with pictures of persons although the dyslexics learned to associate familiar names with pictures as quickly as the controls did. The acquisition of new phonological representations of words was studied in an imitation task with maximally distinct pronunciations of long, familiar words. The dyslexics gained less than the controls in this task. They also gained less on one measure taken from a phoneme substitution task with the same words as in the distinctness task. The results are interpreted in the light of the hypothesis that poorly specified phonological representations may be an underlying problem in dyslexia.

Awareness↗

PubMed2007

Representing word meaning and order information in a composite holographic lexicon.

The authors present a computational model that builds a holographic lexicon representing both word meaning and word order from unsupervised experience with natural language. The model uses simple convolution and superposition mechanisms (cf. B. B. Murdock, 1982) to learn distributed holographic representations for words. The structure of the resulting lexicon can account for empirical data from classic experiments studying semantic typicality, categorization, priming, and semantic constraint in sentence completions. Furthermore, order information can be retrieved from the holographic representations, allowing the model to account for limited word transitions without the need for built-in transition rules. The model demonstrates that a broad range of psychological data can be accounted for directly from the structure of lexical representations learned in this way, without the need for complexity to be built into either the processing mechanisms or the representations. The holographic representations are an appropriate knowledge representation to be used by higher order models of language comprehension, relieving the complexity required at the higher level.

Association↗

PubMed2001

Learning object representations using a priori constraints within ORASSYLL.

In this article, a biologically plausible and efficient object recognition system (called ORASSYLL) is introduced, based on a set of a priori constraints motivated by findings of developmental psychology and neurophysiology. These constraints are concerned with the organization of the input in local and corresponding entities, the interpretation of the input by its transformation in a highly structured feature space, and the evaluation of features extracted from an image sequence by statistical evaluation criteria. In the context of the bias-variance dilemma, the functional role of a priori knowledge within ORASSYLL is discussed. In contrast to systems in which object representations are defined manually,the introduced constraints allow an autonomous learning from complex scenes.

Artificial Intelligence↗

PubMed2001

Conjunctive representations in learning and memory: principles of cortical and hippocampal function.

The authors present a theoretical framework for understanding the roles of the hippocampus and neocortex in learning and memory. This framework incorporates a theme found in many theories of hippocampal function: that the hippocampus is responsible for developing conjunctive representations binding together stimulus elements into a unitary representation that can later be recalled from partial input cues. This idea is contradicted by the fact that hippocampally lesioned rats can learn nonlinear discrimination problems that require conjunctive representations. The authors' framework accommodates this finding by establishing a principled division of labor, where the cortex is responsible for slow learning that integrates over multiple experiences to extract generalities whereas the hippocampus performs rapid learning of the arbitrary contents of individual experiences. This framework suggests that tasks involving rapid, incidental conjunctive learning are better tests of hippocampal function. The authors implement this framework in a computational neural network model and show that it can account for a wide range of data in animal learning.

Animals↗

PubMed1993

On the role of the hippocampus in learning and memory in the rat.

An overview of lesion experiments concerned with the involvement of the hippocampus in learning and memory in the rat is presented. Multiple injections of small amounts of ibotenic acid were used to selectively remove the hippocampus (dentate gyrus, hilar cells, CA1-CA3 pyramidal cells). Similar selective, axon-sparing ibotenate lesions of hippocampus were used in a series of learning and memory experiments employing tasks that are thought to be important in hippocampal function. The performance of rats with the hippocampus removed was compared with that of control animals in the acquisition and retention of spatial versus nonspatial information, forgetting of spatial and nonspatial information, contextual learning, recognition memory and concurrent discrimination learning, and complex representational learning (conditional discrimination and negative patterning learning). The general finding that rats without a hippocampus were impaired on those tasks that required the utilization of spatial and contextual information stands in contrast with the spared performance that was found in learning about and handling (even complex) nonspatial information. Rather than support for views that emphasize a role for the hippocampus in specific memory processes (working memory, declarative memory, temporary memory buffer, configural learning), the present results are more compatible with the idea that the hippocampus plays an especially important role in processing and remembering spatial and contextual information. The limited data that are available using more selective lesions of related hippocampal formation structures (entorhinal cortex, subiculum) suggest that these structures also make important contributions to learning and memory, and that some of these contributions may be different from those made by the hippocampus.

Animals↗

PubMed1995

Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning.

A novel neural network model is presented that learns by trial-and-error to reproduce complex sensory-motor sequences. One subnetwork, corresponding to the prefrontal cortex (PFC), is responsible for generating unique patterns of activity that represent the continuous state of sequence execution. A second subnetwork, corresponding to the striatum, associates these state-encoding patterns with the correct response at each point in the sequence execution. From a neuroscience perspective, the model is based on the known cortical and subcortical anatomy of the primate oculomotor system. From a theoretical perspective, the architecture is similar to that of a finite automaton in which outputs and state transitions are generated as a function of inputs and the current state. Simulation results for complex sequence reproduction and sequence discrimination are presented.

Animals↗

PubMed2000

Learning overcomplete representations.

In an overcomplete basis, the number of basis vectors is greater than the dimensionality of the input, and the representation of an input is not a unique combination of basis vectors. Overcomplete representations have been advocated because they have greater robustness in the presence of noise, can be sparser, and can have greater flexibility in matching structure in the data. Overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex. Previous work has focused on finding the best representation of a signal using a fixed overcomplete basis (or dictionary). We present an algorithm for learning an overcomplete basis by viewing it as probabilistic model of the observed data. We show that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency. This can be viewed as a generalization of the technique of independent component analysis and provides a method for Bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures.

Algorithms↗

PubMed2006

Image denoising via sparse and redundant representations over learned dictionaries.

We address the image denoising problem, where zero-mean white and homogeneous Gaussian additive noise is to be removed from a given image. The approach taken is based on sparse and redundant representations over trained dictionaries. Using the K-SVD algorithm, we obtain a dictionary that describes the image content effectively. Two training options are considered: using the corrupted image itself, or training on a corpus of high-quality image database. Since the K-SVD is limited in handling small image patches, we extend its deployment to arbitrary image sizes by defining a global image prior that forces sparsity over patches in every location in the image. We show how such Bayesian treatment leads to a simple and effective denoising algorithm. This leads to a state-of-the-art denoising performance, equivalent and sometimes surpassing recently published leading alternative denoising methods.

Algorithms↗

PubMed2003

Learning and inference in the brain.

This article is about how the brain data mines its sensory inputs. There are several architectural principles of functional brain anatomy that have emerged from careful anatomic and physiologic studies over the past century. These principles are considered in the light of representational learning to see if they could have been predicted a priori on the basis of purely theoretical considerations. We first review the organisation of hierarchical sensory cortices, paying special attention to the distinction between forward and backward connections. We then review various approaches to representational learning as special cases of generative models, starting with supervised learning and ending with learning based upon empirical Bayes. The latter predicts many features, such as a hierarchical cortical system, prevalent top-down backward influences and functional asymmetries between forward and backward connections that are seen in the real brain. The key points made in article are: (i). hierarchical generative models enable the learning of empirical priors and eschew prior assumptions about the causes of sensory input that are inherent in non-hierarchical models. These assumptions are necessary for learning schemes based on information theory and efficient or sparse coding, but are not necessary in a hierarchical context. Critically, the anatomical infrastructure that may implement generative models in the brain is hierarchical. Furthermore, learning based on empirical Bayes can proceed in a biologically plausible way. (ii). The second point is that backward connections are essential if the processes generating inputs cannot be inverted, or the inversion cannot be parameterised. Because these processes involve many-to-one mappings, are non-linear and dynamic in nature, they are generally non-invertible. This enforces an explicit parameterisation of generative models (i.e. backward connections) to afford recognition and suggests that forward architectures, on their own, are not sufficient for perception. (iii). Finally, non-linearities in generative models, mediated by backward connections, require these connections to be modulatory, so that representations in higher cortical levels can interact to predict responses in lower levels. This is important in relation to functional asymmetries in forward and backward connections that have been demonstrated empirically.

Bayes Theorem↗

PubMed1996

Representations in learning new faces: evidence from prosopagnosia.

We report the performance of a prosopagnosic patient on face learning tasks under different encoding instructions (i.e., levels of processing manipulations). R.J. performs at chance when given no encoding instructions or when given "shallow" encoding instruction to focus on facial features. By contrast, he performs relatively well with "deep" encoding instructions to rate faces in terms of personality traits or when provided with semantic and name information during the study phase. We propose that the improvement associated with deep encoding instructions may be related to the establishment of distinct visually derived and identity-specific semantic codes. The benefit associated with deep encoding in R.J., however, was found to be restricted to the specific view of the face presented at study and did not generalize to other views of the same face. These observations suggest that deep encoding instructions may enhance memory for concrete or pictorial representations of faces in patients with prosopagnosia, but that these patients cannot compensate for the inability to construct abstract structural codes that normally allow faces to be recognized from different orientations. We postulate further that R.J.'s poor performance on face learning tasks may be attributable to excessive reliance on a feature-based left hemisphere face processing system that operates primarily on view-specific representations.

Aged↗

PubMed2005

Reward-spatial view representations and learning in the primate hippocampus.

The primate anterior hippocampus (which corresponds to the rodent ventral hippocampus) receives inputs from brain regions involved in reward processing, such as the amygdala and orbitofrontal cortex. To investigate how this affective input may be incorporated into primate hippocampal function, we recorded neuronal activity while rhesus macaques performed a reward-place association task in which each spatial scene shown on a video monitor had one location that, if touched, yielded a preferred fruit juice reward and a second location that yielded a less-preferred juice reward. Each scene had different locations for the different rewards. Of 312 neurons analyzed in the hippocampus, 18% responded more to the location of the preferred reward in different scenes, and 5% responded to the location of the less-preferred reward. When the locations of the preferred rewards in the scenes were reversed, 60% of 44 hippocampal neurons tested reversed the location to which they responded, showing that the reward-place associations could be altered by new learning in a few trials. The majority (82%) of these 44 hippocampal neurons tested did not respond to reward associations in a visual discrimination, object-reward association task. Thus, the primate hippocampus contains a representation of the reward associations of places "out there" being viewed. By associating places with the rewards available, the concept that the primate hippocampus is involved in object-place event memory is now extended to remembering goals available at different spatial locations. This is an important type of association memory.

Animals↗

PubMed2006

Updating representations of learned scenes.

Two experiments were designed to compare scene recognition reaction time (RT) and accuracy patterns following observer versus scene movement. In Experiment 1, participants memorized a scene from a single perspective. Then, either the scene was rotated or the participants moved (0 degrees -360 degrees in 36 degrees increments) around the scene, and participants judged whether the objects' positions had changed. Regardless of whether the scene was rotated or the observer moved, RT increased with greater angular distance between judged and encoded views. In Experiment 2, we varied the delay (0, 6, or 12 s) between scene encoding and locomotion. Regardless of the delay, however, accuracy decreased and RT increased with angular distance. Thus, our data show that observer movement does not necessarily update representations of spatial layouts and raise questions about the effects of duration limitations and encoding points of view on the automatic spatial updating of representations of scenes.

Cognition↗

PubMed2004

Synaptic transformations underlying highly selective auditory representations of learned birdsong.

Stimulus-specific neuronal responses are a striking characteristic of several sensory systems, although the synaptic mechanisms underlying their generation are not well understood. The songbird nucleus HVC (used here as a proper name) contains projection neurons (PNs) that fire temporally sparse bursts of action potentials to playback of the bird's own song (BOS) but are essentially silent when presented with other acoustical stimuli. To understand how such remarkable stimulus specificity emerges, it is necessary to compare the auditory-evoked responsiveness of the afferents of HVC with synaptic responses in identified HVC neurons. We found that inactivating the interfacial nucleus of the nidopallium (NIf) could eliminate all auditory-evoked subthreshold activity in both HVC PN types, consistent with NIf serving as the major auditory afferent of HVC. Simultaneous multiunit extracellular recordings in NIf and intracellular recordings in HVC revealed that NIf population activity and HVC subthreshold responses were similar in their selectivity for BOS and that NIf spikes preceded depolarizations in all HVC cell types. These results indicate that information about the BOS as well as other auditory stimuli is transmitted synaptically from NIf to HVC. Unlike HVC PNs, however, HVC-projecting NIf neurons fire throughout playback of BOS as well as non-BOS stimuli. Therefore, temporally sparse BOS-evoked firing and enhanced BOS selectivity, manifested as an absence of suprathreshold responsiveness to non-BOS stimuli, emerge in HVC. The transformation to a sparse auditory representation parallels differences in NIf and HVC activity patterns seen during singing, which may point to a common mechanism for encoding sensory and motor representations of song.

Animals↗

PubMed2002

Impact of learning on representation of parts and wholes in monkey inferotemporal cortex.

Here we investigated the impact of visual discrimination training on neuronal responses to parts of images and to whole images in inferotemporal (IT) cortex. Monkeys were trained to discriminate among 'baton' stimuli consisting of discrete top and bottom parts joined by a vertical stem. With separate features at each end, we were able to manipulate the two parts of each baton independently. After training the monkeys, we used single-cell recording to compare neuronal responses to learned and unlearned batons. Responses to learned batons, though not enhanced in strength, were enhanced in selectivity for both individual parts and for whole batons. Whole-baton selectivity arose from a form of conjunctive encoding whereby two parts together exerted a greater influence on neuronal activity than predicted by the additive influence of each part considered individually. These results indicate a possible neural mechanism for holistic or configural effects in expert versus novice observers.

Animals↗

PubMed2001

Neurons with two sites of synaptic integration learn invariant representations.

Neurons in mammalian cerebral cortex combine specific responses with respect to some stimulus features with invariant responses to other stimulus features. For example, in primary visual cortex, complex cells code for orientation of a contour but ignore its position to a certain degree. In higher areas, such as the inferotemporal cortex, translation-invariant, rotation-invariant, and even view point-invariant responses can be observed. Such properties are of obvious interest to artificial systems performing tasks like pattern recognition. It remains to be resolved how such response properties develop in biological systems. Here we present an unsupervised learning rule that addresses this problem. It is based on a neuron model with two sites of synaptic integration, allowing qualitatively different effects of input to basal and apical dendritic trees, respectively. Without supervision, the system learns to extract invariance properties using temporal or spatial continuity of stimuli. Furthermore, top-down information can be smoothly integrated in the same framework. Thus, this model lends a physiological implementation to approaches of unsupervised learning of invariant-response properties.

Algorithms↗