Biomedical subjects

T Heskes

Publications and source records attributed to T Heskes.

6 recordsLinked to original sources

PubMed2000

On 'natural' learning and pruning in multi-layered perceptrons.

Several studies have shown that natural gradient descent for on-line learning is much more efficient than standard gradient descent. In this article, we derive natural gradients in a slightly different manner and discuss implications for batch-mode learning and pruning, linking them to existing algorithms such as Levenberg-Marquardt optimization and optimal brain surgeon. The Fisher matrix plays an important role in all these algorithms. The second half of the article discusses a layered approximation of the Fisher matrix specific to multilayered perceptrons. Using this approximation rather than the exact Fisher matrix, we arrive at much faster "natural" learning algorithms and more robust pruning procedures.

Algorithms↗

PubMed2000

Prediction of bladder outlet obstruction in men with lower urinary tract symptoms using artificial neural networks.

PURPOSE: To evaluate the performance of a backpropagation artificial neural network (ANN) in the diagnosis of men with lower urinary tract symptoms (LUTS) and to compare its performance to that of a traditional linear regression model. MATERIALS AND METHODS: 1903 LUTS patients referred to the University Hospital Nijmegen between 1992 and 1998 received routine investigation, consisting of transrectal ultrasonography of the prostate, serum PSA measurement, assessment of symptoms and quality of life by the International Prostate Symptom Score (IPSS), urinary flowmetry with determination of maximum flow rate (Qmax), voided volume and post-void residual urine and full pressure flow studies (PFS). Using a three-layered backpropagation ANN with three hidden nodes, the outcome of PFS, quantified by the Abrams-Griffiths number (AG-number), was estimated based on all available non-invasive diagnostic test results plus patient age. The performance of the network was quantified using sensitivity, specificity and the area under the ROC-curve (AUC). The results of the neural network approach were compared to those of a linear regression analysis. RESULTS: Prostate volume, Qmax, voided volume and post void residual urine showed substantial predictive value concerning the outcome of PFS. Patient age, PSA-level, IPSS and Quality of life did not add to that prediction. Using a cut-off value in predicted and true AG-numbers of 40 cm. H2O, the neural network approach yielded sensitivity and specificity of 71% and 69%, respectively. The AUC of the network was 0.75 (standard error = 0.01). A linear regression model produced identical results. CONCLUSIONS: This study shows that at an individual level, the outcome of PFS cannot be predicted accurately by the available non-invasive tests. The use of ANNs, which are better able than traditional regression models to identify non-linear relations and complex interactions between variables, did not improve the prediction of BOO. Thus, if precise urodynamic information is considered important in the diagnosis of men with LUTS, PFS must be carried out. Both neural networks and regression analysis appear promising to identify patients who should undergo PFS, and those in whom PFS can safely be omitted. Furthermore, the ability of ANNs and regression models to predict treatment result should be evaluated.

Humans↗

PubMed1999

Pruning using parameter and neuronal metrics.

In this article, we introduce a measure of optimality for architecture selection algorithms for neural networks: the distance from the original network to the new network in a metric defined by the probability distributions of all possible networks. We derive two pruning algorithms, one based on a metric in parameter space and the other based on a metric in neuron space, which are closely related to well-known architecture selection algorithms, such as GOBS. Our framework extends the theoretically range of validity of GOBS and therefore can explain results observed in previous experiments. In addition, we give some computational improvements for these algorithms.

Algorithms↗

PubMed1999

Partial retraining: a new approach to input relevance determination.

In this article we introduce partial retraining, an algorithm to determine the relevance of the input variables of a trained neural network. We place this algorithm in the context of other approaches to relevance determination. Numerical experiments on both artificial and real-world problems show that partial retraining outperforms its competitors, which include methods based on constant substitution, analysis of weight magnitudes, and "optimal brain surgeon".

Algorithms↗

PubMed1998

Bias/Variance Decompositions for Likelihood-Based Estimators.

The bias/variance decomposition of mean-squared error is well understood and relatively straightforward. In this note, a similar simple decomposition is derived, valid for any kind of error measure that, when using the appropriate probability model, can be derived from a Kullback-Leibler divergence or log-likelihood.

Journal Article↗

PubMed1996

How dependencies between successive examples affect on-line learning.

We study the dynamics of on-line learning for a large class of neural networks and learning rules, including backpropagation for multilayer perceptrons. In this paper, we focus on the case where successive examples are dependent, and we analyze how these dependencies affect the learning process. We define the representation error and the prediction error. The representation error measures how well the environment is represented by the network after learning. The prediction error is the average error that a continually learning network makes on the next example. In the neighborhood of a local minimum of the error surface, we calculate these errors. We find that the more predictable the example presentation, the higher the representation error, i.e., the less accurate the asymptotic representation of the whole environment. Furthermore we study the learning process in the presence of a plateau. Plateaus are flat spots on the error surface, which can severely slow down the learning process. In particular, they are notorious in applications with multilayer perceptrons. Our results, which are confirmed by simulations of a multilayer perceptron learning a chaotic time series using backpropagation, explain how dependencies between examples can help the learning process to escape from a plateau.

Learning↗