Biomedical subjects

A F Murray

Publications and source records attributed to A F Murray.

6 recordsLinked to original sources

PubMed2000

A study of early stopping and model selection applied to the papermaking industry.

This paper addresses the issues of neural network model development and maintenance in the context of a complex task taken from the papermaking industry. In particular, it describes a comparison study of early stopping techniques and model selection, both to optimise neural network models for generalisation performance. The results presented here show that early stopping via use of a Bayesian model evidence measure is a viable way of optimising performance while also making maximum use of all the data. In addition, they show that ten-fold cross-validation performs well as a model selector and as an estimator of prediction accuracy. These results are important in that they show how neural network models may be optimally trained and selected for highly complex industrial tasks where the data are noisy and limited in number.

Algorithms↗

PubMed1998

Toward optimally distributed computation.

This article introduces the concept of optimally distributed computation in feedforward neural networks via regularization of weight saliency. By constraining the relative importance of the parameters, computation can be distributed thinly and evenly throughout the network. We propose that this will have beneficial effects on fault-tolerance performance and generalization ability in large network architectures. These theoretical predictions are verified by simulation experiments on two problems: one artificial and the other a real-world task. In summary, this article presents regularization terms for distributing neural computation optimally.

Computer Simulation↗

PubMed1995

Can deterministic penalty terms model the effects of synaptic weight noise on network fault-tolerance?

This paper investigates fault tolerance in feedforward neural networks, for a realistic fault model based on analog hardware. In our previous work with synaptic weight noise we showed significant fault tolerance enhancement over standard training algorithms. We proposed that when introduced into training, weight noise distributes the network computation more evenly across the weights and thus enhances fault tolerance. Here we compare those results with an approximation to the mechanisms induced by stochastic weight noise, incorporated into training deterministically via penalty terms. The penalty terms are an approximation to weight saliency and therefore, in addition, we assess a number of other weight saliency measures and perform comparison experiments. The results show that the first term approximation is an incomplete model of weight noise in terms of fault tolerance. Also the error Hessian is shown to be the most accurate measure of weight saliency.

Neural Networks, Computer↗

PubMed1993

Pulse stream VLSI circuits and systems: the EPSILON neural network chipset.

An analogue CMOS VLSI neural processing chip has been designed and fabricated. The device employs "pulse-stream" neural state signalling and is capable of computing some 360 million synaptic connections per second. In addition to basic characterisation results, the performance of the chip in solving "real-world" problems is also demonstrated. The experience gained from the development of this device has resulted in the design of a second "pulse-stream" chip with improved performance and features. It is anticipated that this second device will be integrated into a standard bus-based system and find early application in robotic control.

Computers, Analog↗

PubMed1993

Analogue synaptic noise--implications and learning improvements.

We analyse the effects of analogue noise on the synaptic arithmetic during multilayer perceptron training by expanding the cost function to include noise-mediated penalty terms. Predictions are made in the light of these calculations which suggest that fault tolerance, generalisation ability and learning trajectory should be improved by such noise-injection. Extensive simulation experiments on two distinct classification problems substantiate the claims. The results appear to be perfectly general for all training schemes where weights are adjusted incrementally, and have wide-ranging implications for all applications, particularly those involving "inaccurate" analogue neural VLSI.

Algorithms↗

PubMedDate not supplied

The prospects for analogue neural VLSI.

In recent years, the efforts of analogue, neural-hardware designers have shifted from generic analogue neurocomputers to "niche" markets in sensor fusion and robotics, and we explain why this is so. We describe the main differences between digital and analogue computation, and consider the advantages of pure analogue and pulsed methods of design. We then investigate some important issues in analogue design of neural machines, namely weight storage (volatile and non-volatile), on-chip learning, and arithmetic accuracy and its relationship to noise. Finally, we outline those areas in which analogue techniques are likely to prove most useful, and speculate as to their likely long-term utility.

Computers, Analog↗