PubMed Health⌕ Search

PubMed · 10937962

Learning non-stationary conditional probability distributions.

Abstract

While sophisticated neural networks and graphical models have been developed for predicting conditional probabilities in a non-stationary environment, major improvements in the training schemes are still required to make these approaches practically viable.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

D Husmeier. 2000. Learning non-stationary conditional probability distributions.. https://doi.org/10.1016/s0893-6080(00)00018-6

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

Balanced state of networks of winner-take-all units.

Irregularly timed action potentials, or spikes, are pervasively observed in the brain activity of awake mammals. However, the role of this temporal irregularity in neural computation is still not well understood. In canonical network models irregular spiking emerges via balanced, fluctuating input currents, leading to collective responses that track inputs linearly. How networks characterized by irregular spiking could support flexible nonlinear dynamics needed for general-purpose computation remains under ongoing debate. Here we characterize the dynamics of networks whose elementary unit is not a single neuron but a small group of neurons, with distinct tunings, that compete at each timestep via a winner-take-all (WTA) interaction. While WTA has long been proposed as an elementary functional motif in the brain and represents a powerful computational primitive, how large networks of such units behave has received less investigation. We show that these networks, like classic excitatory-inhibitory balanced networks, exhibit a chaotic fluctuation-driven regime characterized by sustained irregular activity resembling realistic cortical spiking, which we interpret as a multidimensional balance spread over several competing neural populations with different tunings. We develop a mean-field theory for the network, which shows how irregular spiking sustained by time-varying input fluctuations can support flexible nonlinear collective dynamics. Using the theory we predict and verify network regimes in which input fluctuations alone yield multistability, stable sequence generation, or complex heterogeneous firing rate dynamics-three core dynamical primitives thought to underlie memory-dependent neural computation-via consistent Poisson-like spiking produced through chaos. Thus, networks of WTA units support a chaotic fluctuation-driven regime characterized by irregular spiking that can power complex nonlinear collective dynamics. This represents a new model of brain activity capable of simultaneously reproducing realistic spike trains and diverse nonlinear firing rate patterns well posed for flexible computation, and which can be trained or fit to data.

Models, Neurological↗

Sensitivity of basic oscillatory mechanisms for pattern generation and detection.

Intrinsic oscillators are the basic building blocks of central pattern generators, which model the neural circuits underlying pattern generation. Coupled intrinsic oscillators have been shown to synchronize their oscillatory frequencies and to maintain a characteristic pattern of phase relationships. Recently, oscillatory neurons have also been identified in sensory systems that are involved in decoding phase information. It has been hypothesized that the neural oscillators are part of neural circuits that implement phase-locked loops (PLLs), which are well-known electrical circuits for temporal decoding. Thus, there is evidence that intrinsic neural oscillators participate in both temporal pattern generation and temporal pattern decoding. The present paper investigates the dynamics underlying forced oscillators and forced PLLs, using a single framework, and compares both their stability and sensitivity characteristics. In particular, a method for assessing whether an oscillatory neuron is forced directly or indirectly, as part of a PLL, is developed and applied to published data.

Models, Neurological↗

The timing game.

In two publications presented as a contest, Hopfield and Brody propose a mechanism for detecting groups of neurons receiving similar levels of sensory input.

Models, Neurological↗