PubMed Health⌕ Search

SEARCH · PubMed Health

Results for “Gaussian process”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 recordsLinked to original sources

Predictive approaches for choosing hyperparameters in gaussian processes.

Gaussian processes are powerful regression models specified by parameterized mean and covariance functions. Standard approaches to choose these parameters (known by the name hyperparameters) are maximum likelihood and maximum a posteriori. In this article, we propose and investigate predictive approaches based on Geisser's predictive sample reuse (PSR) methodology and the related Stone's cross-validation (CV) methodology. More specifically, we derive results for Geisser's surrogate predictive probability (GPP), Geisser's predictive mean square error (GPE), and the standard CV error and make a comparative study. Within an approximation we arrive at the generalized cross-validation (GCV) and establish its relationship with the GPP and GPE approaches. These approaches are tested on a number of problems. Experimental results show that these approaches are strongly competitive with the existing approaches.

Likelihood Functions↗

Quantitative structure-activity relationship studies using Gaussian processes.

A Gaussian process method (GPM) is described and applied to the production of some QSAR models. These models have the potential to solve a number of problems which arise in QSAR modeling in that no parameters have to be supplied and only one hyperparameter is used in finding the optimal solution. The application of the method to QSAR is illustrated using data sets of compounds active at the benzodiazepine and muscarinic receptors as well as the data set of the toxicity of substituted benzenes to the ciliate, Tetrahymena Pyriformis.

Journal Article↗

Gaussian processes for machine learning.

Gaussian processes (GPs) are natural generalisations of multivariate Gaussian random variables to infinite (countably or continuous) index sets. GPs have been applied in a large number of fields to a diverse range of ends, and very many deep theoretical analyses of various properties are available. This paper gives an introduction to Gaussian processes on a fairly elementary level with special emphasis on characteristics relevant in machine learning. It draws explicit connections to branches such as spline smoothing models and support vector machines in which similar ideas have been investigated. Gaussian process models are routinely used to solve hard machine learning problems. They are attractive because of their flexible non-parametric nature and computational simplicity. Treated within a Bayesian framework, very powerful statistical methods can be implemented which offer valid estimates of uncertainties in our predictions and generic model selection procedures cast as nonlinear optimization problems. Their main drawback of heavy computational scaling has recently been alleviated by the introduction of generic sparse approximations.13,78,31 The mathematical literature on GPs is large and often uses deep concepts which are not required to fully understand most machine learning applications. In this tutorial paper, we aim to present characteristics of GPs relevant to machine learning and to show up precise connections to other "kernel machines" popular in the community. Our focus is on a simple presentation, but references to more detailed sources are provided.

Algorithms↗

Implementing Gaussian process inference with neural networks.

Gaussian processes compare favourably with backpropagation neural networks as a tool for regression, and Bayesian neural networks have Gaussian process behaviour when the number of hidden neurons tends to infinity. We describe a simple recurrent neural network with connection weights trained by one-shot Hebbian learning. This network amounts to a dynamical system which relaxes to a stable state in which it generates predictions identical to those of Gaussian process regression. In effect an infinite number of hidden units in a feed-forward architecture can be replaced by a merely finite number, together with recurrent connections.

Algorithms↗

Bayesian Gaussian process classification with the EM-EP algorithm.

Gaussian process classifiers (GPCs) are Bayesian probabilistic kernel classifiers. In GPCs, the probability of belonging to a certain class at an input location is monotonically related to the value of some latent function at that location. Starting from a Gaussian process prior over this latent function, data are used to infer both the posterior over the latent function and the values of hyperparameters to determine various aspects of the function. Recently, the expectation propagation (EP) approach has been proposed to infer the posterior over the latent function. Based on this work, we present an approximate EM algorithm, the EM-EP algorithm, to learn both the latent function and the hyperparameters. This algorithm is found to converge in practice and provides an efficient Bayesian framework for learning hyperparameters of the kernel. A multiclass extension of the EM-EP algorithm for GPCs is also derived. In the experimental results, the EM-EP algorithms are as good or better than other methods for GPCs or Support Vector Machines (SVMs) with cross-validation.

Algorithms↗

Self-similar Gaussian processes for modeling anomalous diffusion.

We study some Gaussian models for anomalous diffusion, which include the time-rescaled Brownian motion, two types of fractional Brownian motion, and models associated with fractional Brownian motion based on the generalized Langevin equation. Gaussian processes associated with these models satisfy the anomalous diffusion relation which requires the mean-square displacement to vary with t(alpha), 0<alpha<2. However, these processes have different properties, thus indicating that the anomalous diffusion relation with a single parameter is insufficient to characterize the underlying mechanism. Although the two versions of fractional Brownian motion and time-rescaled Brownian motion all have the same probability distribution function, the Slepian theorem can be used to compare their first passage time distributions, which are different. Finally, in order to model anomalous diffusion with a variable exponent alpha(t) it is necessary to consider the multifractional extensions of these Gaussian processes.

Journal Article↗

Gaussian process approach to spiking neurons for inhomogeneous Poisson inputs.

This article presents a new theoretical framework to consider the dynamics of a stochastic spiking neuron model with general membrane response to input spike. We assume that the input spikes obey an inhomogeneous Poisson process. The stochastic process of the membrane potential then becomes a gaussian process. When a general type of the membrane response is assumed, the stochastic process becomes a Markov-gaussian process. We present a calculation method for the membrane potential density and the firing probability density. Our new formulation is the extension of the existing formulation based on diffusion approximation. Although the single Markov assumption of the diffusion approximation simplifies the stochastic process analysis, the calculation is inaccurate when the stochastic process involves a multiple Markov property. We find that the variation of the shape of the membrane response, which has often been ignored in existing stochastic process studies, significantly affects the firing probability. Our approach can consider the reset effect, which has been difficult to deal with by analysis based on the first passage time density.

Action Potentials↗

Decomposition of nonlinear non-Gaussian process and its application to nonlinear filter and predictor design.

An approach for decomposing of a Nonlinear Non-Gaussian Process (NNGP) is presented. A set of adjoin processes alpha's are first constructed based on the orthogonal principle so that the linear and nonlinear part of the process can be completely separated by a correlation operation without the statistical assumption on the process (i.e., it is not necessarily a Gaussian Process). The linear and nonlinear filters or predictors can then be designed and implemented independently and the consistency of parameters is guaranteed. An algorithm is given for a second order nonlinear process, and it can easily be extended to higher order cases if necessary. The method is first demonstrated by applying it to a nonlinear filter design problem, i.e., system identification. Finally, the necessity of a proposed decomposition procedure is proven by applying it to an example in which the parameters of a signal model are extracted from a version which is distorted due to the nonlinearity of the channel.

Algorithms↗

Sparse on-line gaussian processes.

We develop an approach for sparse representations of gaussian process (GP) models (which are Bayesian types of kernel machines) in order to overcome their limitations for large data sets. The method is based on a combination of a Bayesian on-line algorithm, together with a sequential construction of a relevant subsample of the data that fully specifies the prediction of the GP model. By using an appealing parameterization and projection techniques in a reproducing kernel Hilbert space, recursions for the effective parameters and a sparse gaussian approximation of the posterior process are obtained. This allows for both a propagation of predictions and Bayesian error measures. The significance and robustness of our approach are demonstrated on a variety of experiments.

Journal Article↗

Inference from accelerated degradation and failure data based on Gaussian process models.

An important problem in reliability and survival analysis is that of modeling degradation together with any observed failures in a life test. Here, based on a continuous cumulative damage approach with a Gaussian process describing degradation, a general accelerated test model is presented in which failure times and degradation measures can be combined for inference about system lifetime. Some specific models when the drift of the Gaussian process depends on the acceleration variable are discussed in detail. Illustrative examples using simulated data as well as degradation data observed in carbon-film resistors are presented.

Carbon↗

Gaussian process modeling in conjunction with individual patient simulation modeling: a case study describing the calculation of cost-effectiveness ratios for the treatment of established osteoporosis.

Individual patient-level models can simulate more complex disease processes than cohort-based approaches. However, large numbers of patients need to be simulated to reduce 1st-order uncertainty, increasing the computational time required and often resulting in the inability to perform extensive sensitivity analyses. A solution, employing Gaussian process techniques, is presented using a case study, evaluating the cost-effectiveness of a sample of treatments for established osteoporosis. The Gaussian process model accurately formulated a statistical relationship between the inputs to the individual patient model and its outputs. This model reduced the time required for future runs from 150 min to virtually-instantaneous, allowing probabilistic sensitivity analyses-to be undertaken. This reduction in computational time was achieved with minimal loss in accuracy. The authors believe that this case study demonstrates the value of this technique in handling 1st- and 2nd-order uncertainty in the context of health economic modeling, particularly when more widely used techniques are computationally expensive or are unable to accurately model patient histories.

Alendronate↗

Complicated substructure from simple circularly symmetric Gaussian processes within the centers of goldfish ganglion cell receptive fields.

The center of the receptive field of some retinal ganglion cells exhibits an interesting fine structure: the relative amplitudes of responses to onset and responses to offset of a small spot of light varies systematically as the spot is positioned at various places within the center. Although this pattern may appear complicated, a simple model can account for it in detail. The model postulates that the ganglion cell receives input from separate ON and OFF processes within the center of its receptive field. These processes have the form of Gaussian functions and are laterally displaced from each other. These central ON and OFF input processes are not associated with the additional antagonistic surround of the receptive field. The model is examined for various parameters of the input processes. The observed systematic variation in the ratio of offset to onset responses is predicted when the two processes are of nearly equal width (standard deviation of the Gaussians). Receptive fields made of more than two Gaussians produce various patterns, depending on the relative standard deviations of the Gaussians. Oblong fields, reminiscent of those found in visual cortex, may be generated from a relatively small number of circularly symmetric Gaussian processes.

Animals↗

Gaussian process: an efficient technique to solve quantitative structure-property relationship problems.

We introduce the Gaussian process (GP) model for the empirical modelling of the log P values of 44 1,2-dithiole-3-one molecules. A brief theoretical description of the method is given. Descriptive and predictive abilities of GP are evaluated and compared to multilinear regression results. Special attention is devoted to the automatic relevance determination (ARD) to reduce input variable numbers, which avoid the use of principal component analysis. The present approach was found to be an efficient method and a good alternative to more complicated using artificial neural network systems.

Models, Chemical↗

Gaussian processes for classification: mean-field algorithms.

We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler "naive" mean-field theory and support vector machines (SVMs) as limiting cases. For both mean-field algorithms and support vector machines, simulation results for three small benchmark data sets are presented. They show that one may get state-of-the-art performance by using the leave-one-out estimator for model selection and the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The second result is taken as strong support for the internal consistency of the mean-field approach.

Algorithms↗

Learning curves for Gaussian process regression: approximations and bounds.

We consider the problem of calculating learning curves (i.e., average generalization performance) of gaussian processes used for regression. On the basis of a simple expression for the generalization error, in terms of the eigenvalue decomposition of the covariance function, we derive a number of approximation schemes. We identify where these become exact and compare with existing bounds on learning curves; the new approximations, which can be used for any input space dimension, generally get substantially closer to the truth. We also study possible improvements to our approximations. Finally, we use a simple exactly solvable learning scenario to show that there are limits of principle on the quality of approximations and bounds expressible solely in terms of the eigenvalue spectrum of the covariance function.

Journal Article↗

Perfusion quantification using Gaussian process deconvolution.

The quantification of perfusion using dynamic susceptibility contrast MRI (DSC-MRI) requires deconvolution to obtain the residual impulse response function (IRF). In this work, a method using the Gaussian process for deconvolution (GPD) is proposed. The fact that the IRF is smooth is incorporated as a constraint in the method. The GPD method, which automatically estimates the noise level in each voxel, has the advantage that model parameters are optimized automatically. The GPD is compared to singular value decomposition (SVD) using a common threshold for the singular values, and to SVD using a threshold optimized according to the noise level in each voxel. The comparison is carried out using artificial data as well as data from healthy volunteers. It is shown that GPD is comparable to SVD with a variable optimized threshold when determining the maximum of the IRF, which is directly related to the perfusion. GPD provides a better estimate of the entire IRF. As the signal-to-noise ratio (SNR) increases or the time resolution of the measurements increases, GPD is shown to be superior to SVD. This is also found for large distribution volumes.

Cerebrovascular Circulation↗

Biomarker discovery in microarray gene expression data with Gaussian processes.

MOTIVATION: In clinical practice, pathological phenotypes are often labelled with ordinal scales rather than binary, e.g. the Gleason grading system for tumour cell differentiation. However, in the literature of microarray analysis, these ordinal labels have been rarely treated in a principled way. This paper describes a gene selection algorithm based on Gaussian processes to discover consistent gene expression patterns associated with ordinal clinical phenotypes. The technique of automatic relevance determination is applied to represent the significance level of the genes in a Bayesian inference framework. RESULTS: The usefulness of the proposed algorithm for ordinal labels is demonstrated by the gene expression signature associated with the Gleason score for prostate cancer data. Our results demonstrate how multi-gene markers that may be initially developed with a diagnostic or prognostic application in mind are also useful as an investigative tool to reveal associations between specific molecular and cellular events and features of tumour physiology. Our algorithm can also be applied to microarray data with binary labels with results comparable to other methods in the literature.

Algorithms↗

Bayesian framework for least-squares support vector machine classifiers, gaussian processes, and kernel Fisher discriminant analysis.

The Bayesian evidence framework has been successfully applied to the design of multilayer perceptrons (MLPs) in the work of MacKay. Nevertheless, the training of MLPs suffers from drawbacks like the nonconvex optimization problem and the choice of the number of hidden units. In support vector machines (SVMs) for classification, as introduced by Vapnik, a nonlinear decision boundary is obtained by mapping the input vector first in a nonlinear way to a high-dimensional kernel-induced feature space in which a linear large margin classifier is constructed. Practical expressions are formulated in the dual space in terms of the related kernel function, and the solution follows from a (convex) quadratic programming (QP) problem. In least-squares SVMs (LS-SVMs), the SVM problem formulation is modified by introducing a least-squares cost function and equality instead of inequality constraints, and the solution follows from a linear system in the dual space. Implicitly, the least-squares formulation corresponds to a regression formulation and is also related to kernel Fisher discriminant analysis. The least-squares regression formulation has advantages for deriving analytic expressions in a Bayesian evidence framework, in contrast to the classification formulations used, for example, in gaussian processes (GPs). The LS-SVM formulation has clear primal-dual interpretations, and without the bias term, one explicitly constructs a model that yields the same expressions as have been obtained with GPs for regression. In this article, the Bayesian evidence framework is combined with the LS-SVM classifier formulation. Starting from the feature space formulation, analytic expressions are obtained in the dual space on the different levels of Bayesian inference, while posterior class probabilities are obtained by marginalizing over the model parameters. Empirical results obtained on 10 public domain data sets show that the LS-SVM classifier designed within the Bayesian evidence framework consistently yields good generalization performances.

Artificial Intelligence↗