A new discriminative kernel from probabilistic models.

Koji Tsuda; Motoaki Kawanabe; Gunnar Rätsch; Sören Sonnenburg; Klaus-Robert Müller

doi:10.1162/08997660260293274

Home/Archive/PubMed

PubMed · 12396568

A new discriminative kernel from probabilistic models.

Koji Tsuda·Motoaki Kawanabe·Gunnar Rätsch·Sören Sonnenburg·Klaus-Robert Müller

View original source ↗Export metadata ↓

Abstract

Recently, Jaakkola and Haussler (1999) proposed a method for constructing kernel functions from probabilistic models. Their so-called Fisher kernel has been combined with discriminative classifiers such as support vector machines and applied successfully in, for example, DNA and protein analysis. Whereas the Fisher kernel is calculated from the marginal log-likelihood, we propose the TOP kernel derived; from tangent vectors of posterior log-odds. Furthermore, we develop a theoretical framework on feature extractors from probabilistic models and use it for analyzing the TOP kernel. In experiments, our new discriminative TOP kernel compares favorably to the Fisher kernel.

Explore related subjects

Models, Neurological Models, Statistical Neural Networks, Computer Proteins Journal Article Research Support, Non-U.S. Gov't

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Koji Tsuda, Motoaki Kawanabe, Gunnar Rätsch, Sören Sonnenburg, Klaus-Robert Müller. 2002. A new discriminative kernel from probabilistic models.. https://doi.org/10.1162/08997660260293274

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2025

Balanced state of networks of winner-take-all units.

Irregularly timed action potentials, or spikes, are pervasively observed in the brain activity of awake mammals. However, the role of this temporal irregularity in neural computation is still not well understood. In canonical network models irregular spiking emerges via balanced, fluctuating input currents, leading to collective responses that track inputs linearly. How networks characterized by irregular spiking could support flexible nonlinear dynamics needed for general-purpose computation remains under ongoing debate. Here we characterize the dynamics of networks whose elementary unit is not a single neuron but a small group of neurons, with distinct tunings, that compete at each timestep via a winner-take-all (WTA) interaction. While WTA has long been proposed as an elementary functional motif in the brain and represents a powerful computational primitive, how large networks of such units behave has received less investigation. We show that these networks, like classic excitatory-inhibitory balanced networks, exhibit a chaotic fluctuation-driven regime characterized by sustained irregular activity resembling realistic cortical spiking, which we interpret as a multidimensional balance spread over several competing neural populations with different tunings. We develop a mean-field theory for the network, which shows how irregular spiking sustained by time-varying input fluctuations can support flexible nonlinear collective dynamics. Using the theory we predict and verify network regimes in which input fluctuations alone yield multistability, stable sequence generation, or complex heterogeneous firing rate dynamics-three core dynamical primitives thought to underlie memory-dependent neural computation-via consistent Poisson-like spiking produced through chaos. Thus, networks of WTA units support a chaotic fluctuation-driven regime characterized by irregular spiking that can power complex nonlinear collective dynamics. This represents a new model of brain activity capable of simultaneously reproducing realistic spike trains and diverse nonlinear firing rate patterns well posed for flexible computation, and which can be trained or fit to data.

Models, Neurological↗

PubMed2003

Spectral phase estimates in the setting of multidirectional coupling.

In this paper we explore conventional techniques for estimating the time lag between two signals, using both time and frequency domain measures. We generate test signals to illustrate that in general, estimated temporal relationships between neural sites connected by multiple pathways are ambiguous except in special cases where multidirectional flow of information can be ruled out a priori. Signals are analyzed using the cross correlogram in the time domain, and coherence, phase and directed transfer function (DTF) in the frequency domain. In particular, we show how similar phase and DTF measures can be obtained from qualitatively different dynamical situations (uni and bi-directional flow), and argue that although the phase and DTF combined provide more directionality information than phase alone, one still needs to observe caution if one suspects multidirectional flow.

Models, Neurological↗

PubMed2003

Contribution of synaptic depression to phase maintenance in a model rhythmic network.

In many rhythmic neuronal networks that operate in a wide range of frequencies, the time of neuronal firing relative to the cycle period (the phase) is invariant. This invariance suggests that when frequency changes, firing time is precisely adjusted either by intrinsic or synaptic mechanisms. We study the maintenance of phase in a computational model in which an oscillator neuron (O) inhibits a follower neuron (F) by comparing the dependency of phase on cycle period in two cases: when the inhibitory synapse is depressing and when it is nondepressing. Of the numerous ways of changing the cycle period, we focus on three cases where either the duration of the active state, the inactive state, or the duty cycle of neuron O remains constant. In each case, we measure the phase at which neuron F fires with respect to the onset of firing in neuron O. With a nondepressing synapse, this phase is generally a monotonic function of cycle period except in a small parameter range in the case of the constant inactive duration. In contrast, with a depressing synapse, there is always a parameter regime in which phase is a cubic function of cycle period: it decreases at short cycle periods, increases in an intermediate range, and decreases at long cycle periods. This complex shape for the phase-period relationship arises because of the interaction between synaptic dynamics and intrinsic properties of the postsynaptic neuron. By choosing appropriate parameters, the cubic shape of the phase-period curve results in a small variation in phase for a large interval of periods. Consequently, we find that although a depressing synapse does not produce perfect phase maintenance, in most cases it is superior to a nondepressing synapse in promoting a constant phase difference.

Models, Neurological↗