SEARCH · PubMed Health

Results for “Multilayer Perceptrons”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12Linked to original sources

PubMed2004

Using the EM algorithm to train neural networks: misconceptions and a new algorithm for multiclass classification.

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.

Algorithms↗

PubMed2004

Feature selection in MLPs and SVMs based on maximum output information.

This paper presents feature selection algorithms for multilayer perceptrons (MLPs) and multiclass support vector machines (SVMs), using mutual information between class labels and classifier outputs, as an objective function. This objective function involves inexpensive computation of information measures only on discrete variables; provides immunity to prior class probabilities; and brackets the probability of error of the classifier. The maximum output information (MOI) algorithms employ this function for feature subset selection by greedy elimination and directed search. The output of the MOI algorithms is a feature subset of user-defined size and an associated trained classifier (MLP/SVM). These algorithms compare favorably with a number of other methods in terms of performance on various artificial and real-world data sets.

Algorithms↗

PubMed2004

Temporal processing in primate motor control: relation between cortical and EMG activity.

We investigated spatio-temporal information processing in the primate motor system. Corticomotoneuronal (CM) cells provide monosynaptic excitatory connections from motor cortex to spinal motoneurons and contribute causally to the time-varying electromyogram (EMG) of their target muscle. A multilayer perceptron (MLP) was used to evaluate the transfer function between neural activity of single CM cells and their target muscle EMG, using data from in-vivo recordings in primate motor cortex. For an optimal MLP performance, i.e., minimal error between recorded target EMG and MLP-derived EMG, the CM cell input period had to span the latency observed between CM cell peak activity and EMG peak activity. We argue that the same spike train may code two types of information: 1) rate coding within the input window accounted for large-amplitude variations in the EMG signal and 2) temporal coding within a window of 40 ms just prior to the EMG output signal accounted for EMG variations of small amplitude. The transfer function of the MLP, thus, combines rate and temporal coding and suggests that CM cell output may also combine these two forms of coding. We predict that mutual constraints of rate and temporal coding would, however, would limit the CM output to code for particular temporal profiles of EMG, possibly adapted to bio-mechanical constraints.

Action Potentials↗

PubMed2005

Geometrical interpretation and architecture selection of MLP.

A geometrical interpretation of the multilayer perceptron (MLP) is suggested in this paper. Some general guidelines for selecting the architecture of the MLP, i.e., the number of the hidden neurons and the hidden layers, are proposed based upon this interpretation and the controversial issue of whether four-layered MLP is superior to the three-layered MLP is also carefully examined.

Algorithms↗

PubMed2004

A general framework for learning rules from data.

With the aim of getting understandable symbolic rules to explain a given phenomenon, we split the task of learning these rules from sensory data in two phases: a multilayer perceptron maps features into propositional variables and a set of subsequent layers operated by a PAC-like algorithm learns Boolean expressions on these variables. The special features of this procedure are that: i) the neural network is trained to produce a Boolean output having the principal task of discriminating between classes of inputs; ii) the symbolic part is directed to compute rules within a family that is not known a priori; iii) the welding point between the two learning systems is represented by a feedback based on a suitability evaluation of the computed rules. The procedure we propose is based on a computational learning paradigm set up recently in some papers in the fields of theoretical computer science, artificial intelligence and cognitive systems. The present article focuses on information management aspects of the procedure. We deal with the lack of prior information about the rules through learning strategies that affect both the meaning of the variables and the description length of the rules into which they combine. The paper uses the task of learning to formally discriminate among several emotional states as both a working example and a test bench for a comparison with previous symbolic and subsymbolic methods in the field.

Algorithms↗

PubMed2005

Automated classification and analysis of the calcium response of single T lymphocytes using a neural network approach.

The gene activities in T lymphocytes that regulate immune responses are influenced by Ca2+ ([Ca2+]i). The intracellular calcium signals are highly heterogeneous and vitally important in determining the immune outcome. The signals in individual cells can be measured using fluorescence microscopy but to group the cells into classes with similar signal kinetics is currently laborious. Here, we demonstrate a method for the automated classification of the responses into four categories formerly identified by an expert's inspection. This method comprises characterising the response by a second-order model, performing frequency analysis, and using derived features as inputs to two multilayer perceptron neural networks (NNs). We compare the algorithm's performance on an example data set against the human classification: it was found to classify identically more than 70% of the data, despite small sample sizes in two categories and significant overlap between the other two classes. The group characterized by an oscillating signal showed the presence of a number of frequencies, which may be important in determining gene activation. A classification threshold enables the automatic identification of patterns with a low-classification certainty. Future refinement of the algorithm may allow the identification of more classes, which may be important in different immune responses associated with disease.

Algorithms↗

PubMed2005

Connectionist-based Dempster-Shafer evidential reasoning for data fusion.

Dempster-Shafer evidence theory (DSET) is a popular paradigm for dealing with uncertainty and imprecision. Its corresponding evidential reasoning framework is theoretically attractive. However, there are outstanding issues that hinder its use in real-life applications. Two prominent issues in this regard are 1) the issue of basic probability assignments (masses) and 2) the issue of dependence among information sources. This paper attempts to deal with these issues by utilizing neural networks in the context of pattern classification application. First, a multilayer perceptron neural network with the mean squared error as a cost function is implemented to calculate, for each information source, posteriori probabilities for all classes. Second, an evidence structure construction scheme is developed for transferring the estimated posteriori probabilities to a set of masses along with the corresponding focal elements, from a Bayesian decision point of view. Third, a network realization of the Dempster-Shafer evidential reasoning is designed and analyzed, and it is further extended to a DSET-based neural network, referred to as DSETNN, to manipulate the evidence structures. In order to tackle the issue of dependence between sources, DSETNN is tuned for optimal performance through a supervised learning process. To demonstrate the effectiveness of the proposed approach, we apply it to three benchmark pattern classification problems. Experiments reveal that the DSETNN out-performs DSET and provide encouraging results in terms of classification accuracy and the speed of learning convergence.

Algorithms↗

PubMed2005

Neural and fuzzy computation techniques for playout delay adaptation in VoIP networks.

Playout delay adaptation algorithms are often used in real time voice communication over packet-switched networks to counteract the effects of network jitter at the receiver. Whilst the conventional algorithms developed for silence-suppressed speech transmission focused on preserving the relative temporal structure of speech frames/packets within a talkspurt (intertalkspurt adaptation), more recently developed algorithms strive to achieve better quality by allowing for playout delay adaptation within a talkspurt (intratalkspurt adaptation). The adaptation algorithms, both intertalkspurt and intratalkspurt based, rely on short term estimations of the characteristics of network delay that would be experienced by up-coming voice packets. The use of novel neural networks and fuzzy systems as estimators of network delay characteristics are presented in this paper. Their performance is analyzed in comparison with a number of traditional techniques for both inter and intratalkspurt adaptation paradigms. The design of a novel fuzzy trend analyzer system (FTAS) for network delay trend analysis and its usage in intratalkspurt playout delay adaptation are presented in greater detail. The performance of the proposed mechanism is analyzed based on measured Internet delays. Index Terms-Fuzzy delay trend analysis, intertalkspurt, intratalkspurt, multilayer perceptrons (MLPs), network delay estimation, playout buffering, playout delay adaptation, time delay neural networks (TDNNs), voice over Internet protocol (VoIP).

Algorithms↗

PubMed2006

A spectral clustering approach to underdetermined postnonlinear blind source separation of sparse sources.

This letter proposes a clustering-based approach for solving the underdetermined (i.e., fewer mixtures than sources) postnonlinear blind source separation (PNL BSS) problem when the sources are sparse. Although various algorithms exist for the underdetermined BSS problem for sparse sources, as well as for the PNL BSS problem with as many mixtures as sources, the nonlinear problem in an underdetermined scenario has not been satisfactorily solved yet. The method proposed in this letter aims at inverting the different nonlinearities, thus reducing the problem to linear underdetermined BSS. To this end, first a spectral clustering technique is applied that clusters the mixture samples into different sets corresponding to the different sources. Then, the inverse nonlinearities are estimated using a set of multilayer perceptrons (MLPs) that are trained by minimizing a specifically designed cost function. Finally, transforming each mixture by its corresponding inverse nonlinearity results in a linear underdetermined BSS problem, which can be solved using any of the existing methods.

Algorithms↗

PubMed2006

Accuracy/diversity and ensemble MLP classifier design.

The difficulties of tuning parameters of multilayer perceptrons (MLP) classifiers are well known. In this paper, a measure is described that is capable of predicting the number of classifier training epochs for achieving optimal performance in an ensemble of MLP classifiers. The measure is computed between pairs of patterns on the training data and is based on a spectral representation of a Boolean function. This representation characterizes the mapping from classifier decisions to target label and allows accuracy and diversity to be incorporated within a single measure. Results on many benchmark problems, including the Olivetti Research Laboratory (ORL) face database demonstrate that the measure is well correlated with base-classifier test error, and may be used to predict the optimal number of training epochs. While correlation with ensemble test error is not quite as strong, it is shown in this paper that the measure may be used to predict number of epochs for optimal ensemble performance. Although the technique is only applicable to two-class problems, it is extended here to multiclass through output coding. For the output-coding technique, a random code matrix is shown to give better performance than one-per-class code, even when the base classifier is well-tuned.

Algorithms↗

PubMed2006

Accurate and fast off and online fuzzy ARTMAP-based image classification with application to genetic abnormality diagnosis.

We propose and investigate the fuzzy ARTMAP neural network in off and online classification of fluorescence in situ hybridization image signals enabling clinical diagnosis of numerical genetic abnormalities. We evaluate the classification task (detecting a several abnormalities separately or simultaneously), classifier paradigm (monolithic or hierarchical), ordering strategy for the training patterns (averaging or voting), training mode (for one epoch, with validation or until completion) and model sensitivity to parameters. We find the fuzzy ARTMAP accurate in accomplishing both tasks requiring only very few training epochs. Also, selecting a training ordering by voting is more precise than if averaging over orderings. If trained for only one epoch, the fuzzy ARTMAP provides fast, yet stable and accurate learning as well as insensitivity to model complexity. Early stop of training using a validation set reduces the fuzzy ARTMAP complexity as for other machine learning models but cannot improve accuracy beyond that achieved when training is completed. Compared to other machine learning models, the fuzzy ARTMAP does not loose but gain accuracy when overtrained, although increasing its number of categories. Learned incrementally, the fuzzy ARTMAP reaches its ultimate accuracy very fast obtaining most of its data representation capability and accuracy by using only a few examples. Finally, the fuzzy ARTMAP accuracy for this domain is comparable with those of the multilayer perceptron and support vector machine and superior to those of the naive Bayesian and linear classifiers.

Artificial Intelligence↗

PubMed2006

Evaluation of neural network robust reliability using information-gap theory.

A novel technique for the evaluation of neural network robustness against uncertainty using a nonprobabilistic approach is presented. Conventional optimization techniques were employed to train multilayer perceptron (MLP) networks, which were then probed with an uncertainty analysis using an information-gap model to quantify the network response to uncertainty in the input data. It is demonstrated that the best performing network on data with low uncertainty is not in general the optimal network on data with a higher degree of input uncertainty. Using the concepts of information-gap theory, this paper develops a theoretical framework for information-gap uncertainty applied to neural networks, and explores the practical application of the procedure to three sample cases. The first consists of a simple two-dimensional (2-D) classification network operating on a known Gaussian distribution, the second a nine-lass vibration classification problem from an aircraft wing, and the third a two-class example from a database of breast cancer incidence.

Algorithms↗

PubMed2006

Parameter incremental learning algorithm for neural networks.

In this paper, a novel stochastic (or online) training algorithm for neural networks, named parameter incremental learning (PIL) algorithm, is proposed and developed. The main idea of the PIL strategy is that the learning algorithm should not only adapt to the newly presented input-output training pattern by adjusting parameters, but also preserve the prior results. A general PIL algorithm for feedforward neural networks is accordingly presented as the first-order approximate solution to an optimization problem, where the performance index is the combination of proper measures of preservation and adaptation. The PIL algorithms for the multilayer perceptron (MLP) are subsequently derived. Numerical studies show that for all the three benchmark problems used in this paper the PIL algorithm for MLP is measurably superior to the standard online backpropagation (BP) algorithm and the stochastic diagonal Levenberg-Marquardt (SDLM) algorithm in terms of the convergence speed and accuracy. Other appealing features of the PIL algorithm are that it is computationally as simple as the BP algorithm, and as easy to use as the BP algorithm. It, therefore, can be applied, with better performance, to any situations where the standard online BP algorithm is applicable.

Algorithms↗

PubMed2006

An optimization methodology for neural network weights and architectures.

This paper introduces a methodology for neural network global optimization. The aim is the simultaneous optimization of multilayer perceptron (MLP) network weights and architectures, in order to generate topologies with few connections and high classification performance for any data sets. The approach combines the advantages of simulated annealing, tabu search and the backpropagation training algorithm in order to generate an automatic process for producing networks with high classification performance and low complexity. Experimental results obtained with four classification problems and one prediction problem has shown to be better than those obtained by the most commonly used optimization techniques.

Algorithms↗

PubMed2005

Nonlinear modeling of FES-supported standing-up in paraplegia for selection of feedback sensors.

This paper presents analysis of the standing-up manoeuvre in paraplegia considering the body supportive forces as a potential feedback source in functional electrical stimulation (FES)-assisted standing-up. The analysis investigates the significance of arm, feet, and seat reaction signals to the human body center-of-mass (COM) trajectory reconstruction. The standing-up behavior of eight paraplegic subjects was analyzed, measuring the motion kinematics and reaction forces to provide the data for modeling. Two nonlinear empirical modeling methods are implemented--Gaussian process (GP) priors and multilayer perceptron artificial neural networks (ANN)--and their performance in vertical and horizontal COM component reconstruction is compared. As the input, ten sensory configurations that incorporated different number of sensors were evaluated trading off the modeling performance for variables chosen and ease-of-use in everyday application. For the purpose of evaluation, the root-mean-square difference was calculated between the model output and the kinematics-based COM trajectory. Results show that the force feedback in COM assessment in FES assisted standing-up is comparable alternative to the kinematics measurement systems. It was demonstrated that the GP provided better modeling performance, at higher computational cost. Moreover, on the basis of averaged results, the use of a sensory system incorporating a six-dimensional handle force sensor and an instrumented foot insole is recommended. The configuration is practical for realization and with the GP model achieves an average accuracy of COM estimation 16+/-1.8 mm in horizontal and 39+/-3.7 mm in vertical direction. Some other configurations analyzed in the study exhibit better modeling accuracy, but are less practical for everyday usage.

Adolescent↗

PubMed2005

Skin segmentation using color pixel classification: analysis and comparison.

This paper presents a study of three important issues of the color pixel classification approach to skin segmentation: color representation, color quantization, and classification algorithm. Our analysis of several representative color spaces using the Bayesian classifier with the histogram technique shows that skin segmentation based on color pixel classification is largely unaffected by the choice of the color space. However, segmentation performance degrades when only chrominance channels are used in classification. Furthermore, we find that color quantization can be as low as 64 bins per channel, although higher histogram sizes give better segmentation performance. The Bayesian classifier with the histogram technique and the multilayer perceptron classifier are found to perform better compared to other tested classifiers, including three piecewise linear classifiers, three unimodal Gaussian classifiers, and a Gaussian mixture classifier.

Algorithms↗

PubMed2006

A neural network-based novelty detector for image sequence analysis.

This paper proposes a new model of "novelty detection" for image sequence analysis using neural networks. This model uses the concept of artificially generated negative data to form closed decision boundaries using a multilayer perceptron. The neural network output is novelty filtered by thresholding the output of multiple networks (one per known class) to which the sample is input and clustered for determining which clusters represent novel classes. After labeling these novel clusters, new networks are trained on this data. We perform experiments with video-based image sequence data containing a number of novel classes. The performance of the novelty filter is evaluated using two performance metrics and we compare our proposed model on the basis of these with five baseline novelty detectors. We also discuss the results of retraining each model after novelty detection. On the basis of Chi-square performance metric, we prove at 5 percent significance level that our optimized novelty detector performs at the same level as an ideal novelty detector that does not make any mistakes.

Algorithms↗

PubMed2006

Inducing NNC-Trees with the R4-rule.

An NNC-Tree is a decision tree (DT) with each non-terminal node containing a nearest neighbor classifier (NNC). Compared with the conventional axis-parallel DTs (APDTs), the NNC-Trees can be more efficient, because the decision boundary made by an NNC is more complex than an axis-parallel hyperplane. Compared with single-layer NNCs, the NNC-Trees can classify given data in a hierarchical structure that is often useful for many applications. This paper proposes an algorithm for inducing NNC-Trees based on the R4-rule, which was proposed by the author for finding the smallest nearest neighbor based multilayer perceptrons (NN-MLPs). There are mainly two contributions here. 1) A heuristic but effective method is given to define the teacher signals (group labels) for the data assigned to each nonterminal node. 2) The R4-rule is modified so that an NNC with proper size can be designed automatically in each nonterminal node. Experiments with several public databases show that the proposed algorithm can produce NNC-Trees effectively and efficiently.

Algorithms↗