PubMed Health⌕ Search

Biomedical subjects

G E Hinton

Publications and source records attributed to G E Hinton.

10 recordsLinked to original sources
PubMed1995

The "wake-sleep" algorithm for unsupervised neural networks.

An unsupervised learning algorithm for a multilayer network of stochastic neurons is described. Bottom-up "recognition" connections convert the input into representations in successive hidden layers, and top-down "generative" connections reconstruct the representation in one layer from the representation in the layer above. In the "wake" phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the "sleep" phase, neurons are driven by generative connections, and recognition connections are adapted to increase the probability that they would produce the correct activity vector in the layer above.

Algorithms↗
PubMed1995

The Helmholtz machine.

Discovering the structure inherent in a set of patterns is a fundamental aim of statistical inference or learning. One fruitful approach is to build a parameterized stochastic generative model, independent draws from which are likely to produce the patterns. For all but the simplest generative models, each pattern can be generated in exponentially many ways. It is thus intractable to adjust the parameters to maximize the probability of the observed patterns. We describe a way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations. Our method can be viewed as a form of hierarchical self-supervised learning that may relate to the function of bottom-up and top-down cortical processing pathways.

Algorithms↗
PubMed1992

Self-organizing neural network that discovers surfaces in random-dot stereograms.

The standard form of back-propagation learning is implausible as a model of perceptual learning because it requires an external teacher to specify the desired output of the network. We show how the external teacher can be replaced by internally derived teaching signals. These signals are generated by using the assumption that different parts of the perceptual input have common causes in the external world. Small modules that look at separate but related parts of the perceptual input discover these common causes by striving to produce outputs that agree with each other. The modules may look at different modalities (such as vision and touch), or the same modality at different times (for example, the consecutive two-dimensional views of a rotating three-dimensional object), or even spatially adjacent parts of the same image. Our simulations show that when our learning procedure is applied to adjacent patches of two-dimensional images, it allows a neural network that has no prior knowledge of the third dimension to discovery depth in random dot stereograms of curved surfaces.

Artificial Intelligence↗
PubMed1991

Lesioning an attractor network: investigations of acquired dyslexia.

A recurrent connectionist network was trained to output semantic feature vectors when presented with letter strings. When damaged, the network exhibited characteristics that resembled several of the phenomena found in deep dyslexia and semantic-access dyslexia. Damaged networks sometimes settled to the semantic vectors for semantically similar but visually dissimilar words. With severe damage, a forced-choice decision between categories was possible even when the choice of the particular semantic vector within the category was not possible. The damaged networks typically exhibited many mixed visual and semantic errors in which the output corresponded to a word that was both visually and semantically similar. Surprisingly, damage near the output sometimes caused pure visual errors. Indeed, the characteristic error pattern of deep dyslexia occurred with damage to virtually any part of the network.

Brain↗
PubMed1987

The horizontal-vertical delusion.

Most people can correctly apply the concepts of horizontal and vertical in describing objects, but a simple demonstration shows that they are confused about how these concepts work. The nature of the confusion and its possible causes are briefly discussed.

Humans↗
PubMed1986

Separating figure from ground with a parallel network.

The differentiation of figure from ground plays an important role in the perceptual organization of visual stimuli. The rapidity with which we can discriminate the inside from the outside of a figure suggests that at least this step in the process may be performed in visual cortex by a large number of neurons in several different areas working together in parallel. We have attempted to simulate this collective computation by designing a network of simple processing units that receives two types of information: bottom-up input from the image containing the outlines of a figure, which may be incomplete, and a top-down attentional input that biases one part of the image to be the inside of the figure. No presegmentation of the image was assumed. Two methods for performing the computation were explored: gradient descent, which seeks locally optimal states, and simulated annealing, which attempts to find globally optimal states by introducing noise into the computation. For complete outlines, gradient descent was faster, but the range of input parameters leading to successful performance was very narrow. In contrast, simulated annealing was more robust: it worked over a wider range of attention parameters and a wider range of outlines, including incomplete ones. Our network model is too simplified to serve as a model of human performance, but it does demonstrate that one global property of outlines can be computed through local interactions in a parallel network. Some features of the model, such as the role of noise in escaping from nonglobal optima, may generalize to more realistic models.

Algorithms↗
PubMed1984

Why the islands move.

Micronesian navigators routinely make voyages across large expanses of open ocean. To do this, a navigator must judge both the direction in which he is sailing and the distance he has travelled. The rising and setting points of the stars (and other cues) provide instantaneous information about direction, but distance can only be judged by integrating velocity-related information over time. Micronesian navigators judge distance in a way that seems odd. When they are out of sight of land, they imagine that the canoe is stationary and that the islands move back past them. For each voyage, they 'attend' to an island off to the side of the course which is out of sight over the horizon. As they sail, they imagine the island moving back along the horizon changing in bearing until it is imagined to be under the bearing it is known to have from the destination island. Then they know they are near their destination. There is good reason for using a frame of reference whose origin is defined by the boat. We show how it finesses a perceptual paradox--the rising and setting points of the stars do not exhibit motion parallax.

Cues↗
PubMedDate not supplied

Parallel visual computation.

The functional abilities and parallel architecture of the human visual system are a rich source of ideas about visual processing. Any visual task that we can perform quickly and effortlessly is likely to have a computational solution using a parallel algorithm. Recently, several such parallel algorithms have been found that exploit information implicit in an image to compute intrinsic properties of surfaces, such as surface orientation, reflectance and depth. These algorithms require a computational architecture that has similarities to that of visual cortex in primates.

Computers↗