Biomedical subjects

H J Kappen

Publications and source records attributed to H J Kappen.

7 recordsLinked to original sources

PubMed2001

A tighter bound for graphical models.

We present a method to bound the partition function of a Boltzmann machine neural network with any odd-order polynomial. This is a direct extension of the mean-field bound, which is first order. We show that the third-order bound is strictly better than mean field. Additionally, we derive a third-order bound for the likelihood of sigmoid belief networks. Numerical experiments indicate that an error reduction of a factor of two is easily reached in the region where expansion-based approximations are useful.

Computer Simulation↗

PubMed2000

Learning in higher order Boltzmann machines using linear response.

We introduce an efficient method for learning and inference in higher order Boltzmann machines. The method is based on mean field theory with the linear response correction. We compute the correlations using the exact and the approximated method for a fully connected third order network of ten neurons. In addition, we compare the results of the exact and approximate learning algorithm. Finally we use the presented method to solve the shifter problem. We conclude that the linear response approximation gives good results as long as the couplings are not too large.

Artificial Intelligence↗

PubMed2000

Mean field theory for asymmetric neural networks.

The computation of mean firing rates and correlations is intractable for large neural networks. For symmetric networks one can derive mean field approximations using the Taylor series expansion of the free energy as proposed by Plefka. In asymmetric networks, the concept of free energy is absent. Therefore, it is not immediately obvious how to extend this method to asymmetric networks. In this paper we extend Plefka's approach to asymmetric networks and in fact to arbitrary probability distributions. The method is based on an information geometric argument. The method is illustrated for asymmetric neural networks with sequential dynamics. We compare our approximate analytical results with Monte Carlo simulations for a network of 100 neurons. It is shown that the quality of the approximation for asymmetric networks is as good as for symmetric networks.

Models, Neurological↗

PubMed2000

Nonmonotonic generalization bias of Gaussian mixture models.

Theories of learning and generalization hold that the generalization bias, defined as the difference between the training error and the generalization error, increases on average with the number of adaptive parameters. This article, however, shows that this general tendency is violated for a gaussian mixture model. For temperatures just below the first symmetry breaking point, the effective number of adaptive parameters increases and the generalization bias decreases. We compute the dependence of the neural information criterion on temperature around the symmetry breaking. Our results are confirmed by numerical cross-validation experiments.

Computer Simulation↗

PubMed1999

A development protocol for a diagnostic DSS.

A diagnostic decision support system (DSS) in medicine is an expert system that aids the physician in the determination of the diagnosis based on findings and test results. The DSS can be divided into 2 different types of components: the knowledge component and the information system component. Methods from software engineering, knowledge engineering and management are combined into a dynamic development cycle that allows stepwise update and refinement. The development of the knowledge component is based on knowledge engineering. In the starting phases, rapid prototyping is convenient to determine and evaluate the specification. The content of the knowledge base is frequently updated during its life time. For this purpose, a knowledge modelling protocol is supplied.

Artificial Intelligence↗

PubMed1997

Symmetry breaking and training from incomplete data with Radial Basis Boltzmann Machines.

A Radial Basis Boltzmann Machine (RBBM) is a specialized Boltzmann Machine architecture that combines feed-forward mapping with probability estimation in the input space, and for which very efficient learning rules exist. The hidden representation of the network displays symmetry breaking as a function of the noise in the dynamics. Thus, generalization can be studied as a function of the noise in the neuron dynamics instead of as a function of the number of hidden units. We show that the RBBM can be seen as an elegant alternative of k-nearest neighbor, leading to comparable performance without the need to store all data. We show that the RBBM has good classification performance compared to the MLP. The main advantage of the RBBM is that simultaneously with the input-output mapping, a model of the input space is obtained which can be used for learning with missing values. We derive learning rules for the case of incomplete data, and show that they perform better on incomplete data than the traditional learning rules on a 'repaired' data set.

Computer Simulation↗

PubMed1993

Advanced ovarian cancer. Neural network analysis to predict treatment outcome.

BACKGROUND: Quantitative methods for the analysis of prognostic information are important in order to use this knowledge optimally. The neural network is a new quantitative method where the fundamental building blocks are units which can be likened to neurons, and weighted connections which can be likened to synapses. The more the hidden units, the more complex the patterns that can be learnt. MATERIALS AND METHODS: Data from two Dutch studies in ovarian cancer were used to compare the previously reported survival rates predicted by the Cox's prognostic index with the prediction obtained by a neural network. RESULTS: Both the Cox's analysis and the neural network agreed on residual tumour size, stage, and performance status as being important for survival. The neural network identified additional predictive factors such as place of diagnosis and age. As the Cox's prognostic index has not been tested to predict survival on an independent data set a comparison with the results obtained in the neural network test set could not be performed. CONCLUSIONS: Neural networks perform at least as well as Cox's method for the prediction of survival, and prognostic factors can easily be identified. The analysis not only revealed the predictive power of some characteristics, but also the non-predictive power of the others.

Antineoplastic Combined Chemotherapy Protocols↗