SEARCH · PubMed Health

Results for “Multilayer Perceptrons”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14Linked to original sources

PubMed1999

A space-time delay neural network for motion recognition and its application to lipreading.

Motion recognition has received increasing attention in recent years owing to heightened demand for computer vision in many domains, including the surveillance system, multimodal human computer interface, and traffic control system. Most conventional approaches classify the motion recognition task into partial feature extraction and time-domain recognition subtasks. However, the information of motion resides in the space-time domain instead of the time domain or space domain independently, implying that fusing the feature extraction and classification in the space and time domains into a single framework is preferred. Based on this notion, this work presents a novel Space-Time Delay Neural Network (STDNN) capable of handling the space-time dynamic information for motion recognition. The STDNN is unified structure, in which the low-level spatiotemporal feature extraction and high-level space-time-domain recognition are fused. The proposed network possesses the spatiotemporal shift-invariant recognition ability that is inherited from the time delay neural network (TDNN) and space displacement neural network (SDNN), where TDNN and SDNN are good at temporal and spatial shift-invariant recognition, respectively. In contrast to multilayer perceptron (MLP), TDNN, and SDNN, STDNN is constructed by vector-type nodes and matrix-type links such that the spatiotemporal information can be accurately represented in a neural network. Also evaluated herein is the performance of the proposed STDNN via two experiments. The moving Arabic numerals (MAN) experiment simulates the object's free movement in the space-time domain on image sequences. According to these results, STDNN possesses a good generalization ability with respect to the spatiotemporal shift-invariant recognition. In the lipreading experiment, STDNN recognizes the lip motions based on the inputs of real image sequences. This observation confirms that STDNN yields a better performance than the existing TDNN-based system, particularly in terms of the generalization ability. In addition to the lipreading application, the STDNN can be applied to other problems since no domain-dependent knowledge is used in the experiment.

Algorithms↗

PubMed1999

A new measurement of noise immunity and generalization ability for MLPs.

This paper shows a quantitative relation between the regularization techniques, the generalization ability, and the sensitivity of the Multilayer Perceptron (MLP) to input noise. Although many studies about these topics have been presented, in most cases only one of the problems is addressed, and only experimentally obtained evidence is provided to illustrate some kind of correlation between generalization, noise immunity and the use of regularization techniques to obtain a set of weights after training that provides the corresponding MLP with generalization ability and noise immunity. Here, a new measurement of noise immunity for a MLP is presented. This measurement, which is termed Mean Squared Sensitivity (MSS), explicitly evaluates the Mean Squared Error (MSE) degradation of a MLP when it is perturbed by input noise, and can be computed from the statistical sensitivities (previously proposed) of the output neurons. The MSS provides an accurate evaluation of the MLP performance loss when its inputs are perturbed by noise and can also be considered a measurement of the smoothness of the error surface with respect to the inputs. Thus, as the MSS can be used to evaluate the noise immunity or the generalization ability, it gives a criterion to select among different weight configurations that present a similar MSE after training.

Models, Neurological↗

PubMed2004

[Methodological approach to the use of artificial neural networks for predicting results in medicine].

In clinical practice, there is an increasing interest in obtaining adequate models of prediction. Within the possible available alternatives, the artificial neural networks (ANN) are progressively more used. In this review we first introduce the ANN methodology, describing the most common type of ANN, the Multilayer Perceptron trained with backpropagation algorithm (MLP). Then we compare the MLP with the Logistic Regression (LR). Finally, we show a practical scheme to make an application based on ANN by means of an example with actual data. The main advantage of the RN is its capacity to incorporate nonlinear effects and interactions between the variables of the model without need to include them a priori. As greater disadvantages, they show a difficult interpretation of their parameters and large empiricism in their process of construction and training. ANN are useful for the computation of probabilities of a given outcome based on a set of predicting variables. Furthermore, in some cases, they obtain better results than LR. Both methodologies, ANN and LR, are complementary and they help us to obtain more valid models.

Artificial Intelligence↗

PubMed2005

Delayed disease progression after allogeneic cell vaccination in hormone-resistant prostate cancer and correlation with immunologic variables.

PURPOSE: There are a significant number of patients with asymptomatic hormone-resistant prostate cancer who have increasing prostate-specific antigen (PSA) levels but little or no evaluable disease. The immunogenicity and minimal toxicity associated with cell-based vaccine therapy makes this approach attractive for these patients. EXPERIMENTAL DESIGN: We have evaluated a vaccine comprising monthly intradermal injection of three irradiated allogeneic prostate cell lines (8 x 10(6) cells each) over 1 year. The first two doses were supplemented with bacille Calmette-Guérin as vaccine adjuvant. Twenty-eight hormone-resistant prostate cancer patients were enrolled. Patients were assessed clinically and PSA levels were measured monthly. Radiologic scans (X-ray, computed tomography, and bone scan) were taken at baseline and at intervals throughout the treatment period. Comprehensive monthly immunologic monitoring was undertaken including proliferation studies, activation markers, cytokine protein expression, and gene copy number. This longitudinal data was analyzed through predictive modeling using artificial neural network feed-forward/back-propagation algorithms with multilayer perceptron architecture. RESULTS: Eleven of the 26 patients showed statistically significant, prolonged decreases in their PSA velocity (PSAV). None experienced any significant toxicity. Median time to disease progression was 58 weeks, compared with recent studies of other agents and historical control values of around 28 weeks. PSAV-responding patients showed a titratable T(H)1 cytokine release profile in response to restimulation with a vaccine lysate, while nonresponders showed a mixed T(H)1 and T(H)2 response. Furthermore, immunologic profile correlated with PSAV response by artificial neural network analysis. We found predictive power not only in expression of cytokines after maximal stimulation with phorbol 12-myristate 13-acetate, but also the method of analysis (qPCR measurement of IFN-gamma > qPCR measurement tumor necrosis factor-alpha > protein expression of IFN-gamma > protein expression of interleukin 2). CONCLUSIONS: Whole cell allogeneic vaccination in hormone-resistant prostate cancer is nontoxic and improves the natural history of the disease. Longitudinal changes in immunologic function in vaccinated patients may be better interpreted through predictive modeling using tools such as the artificial neural network rather than periodic "snapshot" readouts.

Aged↗

PubMed2006

Prognostic DNA methylation biomarkers in ovarian cancer.

PURPOSE: Aberrant DNA methylation, now recognized as a contributing factor to neoplasia, often shows definitive gene/sequence preferences unique to specific cancer types. Correspondingly, distinct combinations of methylated loci can function as biomarkers for numerous clinical correlates of ovarian and other cancers. EXPERIMENTAL DESIGN: We used a microarray approach to identify methylated loci prognostic for reduced progression-free survival (PFS) in advanced ovarian cancer patients. Two data set classification algorithms, Significance Analysis of Microarray and Prediction Analysis of Microarray, successfully identified 220 candidate PFS-discriminatory methylated loci. Of those, 112 were found capable of predicting PFS with 95% accuracy, by Prediction Analysis of Microarray, using an independent set of 40 advanced ovarian tumors (from 20 short-PFS and 20 long-PFS patients, respectively). Additionally, we showed the use of these predictive loci using two bioinformatics machine-learning algorithms, Support Vector Machine and Multilayer Perceptron. CONCLUSION: In this report, we show that highly prognostic DNA methylation biomarkers can be successfully identified and characterized, using previously unused, rigorous classifying algorithms. Such ovarian cancer biomarkers represent a promising approach for the assessment and management of this devastating disease.

Adenocarcinoma↗

PubMed2001

Prediction of postoperative prostatic cancer stage on the basis of systematic biopsies using two types of artificial neural networks.

OBJECTIVE: The choice of therapy for prostatic cancer should depend on a rational preoperative estimate of tumor stage. Artificial neural networks were used to predict postoperative staging of prostatic cancer from sextant biopsies and routinely available preoperative data. METHODS: In group I (97 cases), nonorgan confinement (tumor stage > or =pT3a) was predicted on the basis of age and six histopathological variables from sextant biopsies. In group II (77 cases), nonorgan confinement and extraprostatic organ infiltration (tumor classification > or =pT3b) were predicted from age, four histopathological variables, the preoperative PSA level, and the total prostate volume estimated by preoperative ultrasonography. Learning vector quantization (LVQ) networks were applied for this purpose and compared to multilayer perceptrons (MLP) and linear discriminant analysis (LDA). RESULTS: Nonorgan confinement could be predicted correctly in 90% of newly presented cases from sextant biopsy histopathology alone. A similar accuracy of predicting nonorgan confinement (83%) was obtained by combining preoperative biopsy histology with clinical data. Extraprostatic organ infiltration could be predicted correctly in 82%. The best results were obtained by LVQ networks, followed by MLP networks and LDA. CONCLUSION: The postoperative tumor stage of prostatic cancer can be estimated with high accuracy, sensitivity and specificity from preoperative routine parameters using artificial neural networks, especially LVQ networks. The results suggest that this methodology should be evaluated in a larger prospective study.

Aged↗

PubMed2004

Regional cerebral blood flow in Alzheimer's disease: classification and analysis of heterogeneity.

Neural networks have been successfully applied to brain perfusion images to classify patients with Alzheimer's disease from normal or other patient populations. Given the recognition that Alzheimer's disease constitutes a heterogeneous disorder, the identification of subgroups sharing common functional brain deficits would constitute a further improvement in the utility of such methods. Therefore, we aimed to investigate whether neural networks could discriminate cortical perfusion deficits of patients with Alzheimer's disease from normal brain perfusion. A second step was to identify subgroups of patients sharing similar perfusion deficits. The study population consisted of one group of 92 normal healthy subjects and one group of 132 patients with mild-to-moderate Alzheimer's disease. The patients were diagnosed according to established criteria (DSM-IV and NINCDS-ADRDA). Regional cerebral blood flow was assessed by the non-invasive (133)Xe inhalation method, using a 64-detector system for measurements of blood flow in superficial cortical areas. The regional blood flow values were used as the only input to artificial neural networks with multilayer Perceptron architecture. The networks were trained using the back-propagation updating algorithm. A fourfold cross validation procedure was used in order to obtain the most reliable performance of the networks. The performance of the neural network, measured as the area under the receiver-operating characteristic curve, was 0.94, with a sensitivity for Alzheimer's disease of 86% at a specificity of 90%. An analysis of the relative importance of cortical areas in the discrimination showed that left parietal areas were more important than the right homologous ones. A clustering analysis of the Alzheimer patients identified three or four subgroups of patients with clearly different combinations of blood flow pathology. A consistent finding in all subgroups was a significant deficit in temporoparietal blood flow of both hemispheres. Distinct group differences were seen in frontal, central and occipital areas with different combinations of involvement. This is the first study in which neural networks have been applied to brain perfusion images obtained with the (133)Xe inhalation method. The results demonstrate that a classification of patients with Alzheimer's disease obtained with this method is compatible with the best results obtained with other brain imaging methods. The identification of clearly distinguishable patterns of blood flow pathology in subgroups of patients lends further support to the notion that Alzheimer's disease is a heterogeneous disorder.

Adolescent↗

PubMed1997

Comparison between conventional and neural network classifiers for rat sleep-wake stage discrimination.

This article describes an approach to selecting the most efficient classifier for rat sleep staging (waking, REM sleep and NREM sleep) discrimination. Three conventional (bayesian, linear, and euclidean) and two neural network (multilayer perceptrons integrating or not integrating contextual information) classifiers were compared. For each classifier, performances were presented in the form of a statistical concordance matrix comparing classifier results versus human expert results on 6 24-hour records (1 record per animal). Comparison between classifiers was based on the estimation and accuracy of global agreements. Interest was also focused on REM sleep state discrimination. The results show that neural network classifiers are appropriate tools to be integrated in an automatic rat sleep-wake stage system. The approach presented should help scientists in choosing a method of data classification.

Animals↗

PubMed2000

A quantitative study of fault tolerance, noise immunity, and generalization ability of MLPs

An analysis of the influence of weight and input perturbations in a multilayer perceptron (MLP) is made in this article. Quantitative measurements of fault tolerance, noise immunity, and generalization ability are provided. From the expressions obtained, it is possible to justify some previously reported conjectures and experimentally obtained results (e.g., the influence of weight magnitudes, the relation between training with noise and the generalization ability, the relation between fault tolerance and the generalization ability). The measurements introduced here are explicitly related to the mean squared error degradation in the presence of perturbations, thus constituting a selection criterion between different alternatives of weight configurations. Moreover, they allow us to predict the degradation of the learning performance of an MLP when its weights or inputs are deviated from their nominal values and thus, the behavior of a physical implementation can be evaluated before the weights are mapped on it according to its accuracy.

Journal Article↗

PubMed2000

Approximate maximum entropy joint feature inference consistent with arbitrary lower-order probability constraints: application to statistical classification

We propose a new learning method for discrete space statistical classifiers. Similar to Chow and Liu (1968) and Cheeseman (1983), we cast classification/inference within the more general framework of estimating the joint probability mass function (p.m.f.) for the (feature vector, class label) pair. Cheeseman's proposal to build the maximum entropy (ME) joint p.m.f. consistent with general lower-order probability constraints is in principle powerful, allowing general dependencies between features. However, enormous learning complexity has severely limited the use of this approach. Alternative models such as Bayesian networks (BNs) require explicit determination of conditional independencies. These may be difficult to assess given limited data. Here we propose an approximate ME method, which, like previous methods, incorporates general constraints while retaining quite tractable learning. The new method restricts joint p.m.f. support during learning to a small subset of the full feature space. Classification gains are realized over dependence trees, tree-augmented naive Bayes networks, BNs trained by the Kutato algorithm, and multilayer perceptrons. Extensions to more general inference problems are indicated. We also propose a novel exact inference method when there are several missing features.

Journal Article↗

PubMed2000

Hierarchical bayesian models for regularisation in sequential learning

We show that a hierarchical Bayesian modeling approach allows us to perform regularization in sequential learning. We identify three inference levels within this hierarchy: model selection, parameter estimation, and noise estimation. In environments where data arrive sequentially, techniques such as cross validation to achieve regularization or model selection are not possible. The Bayesian approach, with extended Kalman filtering at the parameter estimation level, allows for regularization within a minimum variance framework. A multilayer perceptron is used to generate the extended Kalman filter nonlinear measurements mapping. We describe several algorithms at the noise estimation level that allow us to implement on-line regularization. We also show the theoretical links between adaptive noise estimation in extended Kalman filtering, multiple adaptive learning rates, and multiple smoothing regularization coefficients.

Journal Article↗

PubMed2002

Bayesian model assessment and comparison using cross-validation predictive densities.

In this work, we discuss practical methods for the assessment, comparison, and selection of complex hierarchical Bayesian models. A natural way to assess the goodness of the model is to estimate its future predictive capability by estimating expected utilities. Instead of just making a point estimate, it is important to obtain the distribution of the expected utility estimate because it describes the uncertainty in the estimate. The distributions of the expected utility estimates can also be used to compare models, for example, by computing the probability of one model having a better expected utility than some other model. We propose an approach using cross-validation predictive densities to obtain expected utility estimates and Bayesian bootstrap to obtain samples from their distributions. We also discuss the probabilistic assumptions made and properties of two practical cross-validation methods, importance sampling and k-fold cross-validation. As illustrative examples, we use multilayer perceptron neural networks and gaussian processes with Markov chain Monte Carlo sampling in one toy problem and two challenging real-world problems.

Arm↗

PubMed2002

Bayesian framework for least-squares support vector machine classifiers, gaussian processes, and kernel Fisher discriminant analysis.

The Bayesian evidence framework has been successfully applied to the design of multilayer perceptrons (MLPs) in the work of MacKay. Nevertheless, the training of MLPs suffers from drawbacks like the nonconvex optimization problem and the choice of the number of hidden units. In support vector machines (SVMs) for classification, as introduced by Vapnik, a nonlinear decision boundary is obtained by mapping the input vector first in a nonlinear way to a high-dimensional kernel-induced feature space in which a linear large margin classifier is constructed. Practical expressions are formulated in the dual space in terms of the related kernel function, and the solution follows from a (convex) quadratic programming (QP) problem. In least-squares SVMs (LS-SVMs), the SVM problem formulation is modified by introducing a least-squares cost function and equality instead of inequality constraints, and the solution follows from a linear system in the dual space. Implicitly, the least-squares formulation corresponds to a regression formulation and is also related to kernel Fisher discriminant analysis. The least-squares regression formulation has advantages for deriving analytic expressions in a Bayesian evidence framework, in contrast to the classification formulations used, for example, in gaussian processes (GPs). The LS-SVM formulation has clear primal-dual interpretations, and without the bias term, one explicitly constructs a model that yields the same expressions as have been obtained with GPs for regression. In this article, the Bayesian evidence framework is combined with the LS-SVM classifier formulation. Starting from the feature space formulation, analytic expressions are obtained in the dual space on the different levels of Bayesian inference, while posterior class probabilities are obtained by marginalizing over the model parameters. Empirical results obtained on 10 public domain data sets show that the LS-SVM classifier designed within the Bayesian evidence framework consistently yields good generalization performances.

Artificial Intelligence↗

PubMed2003

Equivalence of backpropagation and contrastive Hebbian learning in a layered network.

Backpropagation and contrastive Hebbian learning are two methods of training networks with hidden neurons. Backpropagation computes an error signal for the output neurons and spreads it over the hidden neurons. Contrastive Hebbian learning involves clamping the output neurons at desired values and letting the effect spread through feedback connections over the entire network. To investigate the relationship between these two forms of learning, we consider a special case in which they are identical: a multilayer perceptron with linear output units, to which weak feedback connections have been added. In this case, the change in network state caused by clamping the output neurons turns out to be the same as the error signal spread by backpropagation, except for a scalar prefactor. This suggests that the functionality of backpropagation can be realized alternatively by a Hebbian-type learning algorithm, which is suitable for implementation in biological networks.

Algorithms↗

PubMed2004

Online adaptive decision trees.

Decision trees and neural networks are widely used tools for pattern classification. Decision trees provide highly localized representation, whereas neural networks provide a distributed but compact representation of the decision space. Decision trees cannot be induced in the online mode, and they are not adaptive to changing environment, whereas neural networks are inherently capable of online learning and adpativity. Here we provide a classification scheme called online adaptive decision trees (OADT), which is a tree-structured network like the decision trees and capable of online learning like neural networks. A new objective measure is derived for supervised learning with OADT. Experimental results validate the effectiveness of the proposed classification scheme. Also, with certain real-life data sets, we find that OADT performs better than two widely used models: the hierarchical mixture of experts and multilayer perceptron.

Computer Simulation↗

PubMed2004

On the asymptotic distribution of the least-squares estimators in unidentifiable models.

In order to analyze the stochastic property of multilayered perceptrons or other learning machines, we deal with simpler models and derive the asymptotic distribution of the least-squares estimators of their parameters. In the case where a model is unidentified, we show different results from traditional linear models: the well-known property of asymptotic normality never holds for the estimates of redundant parameters.

Algorithms↗

PubMed1998

An Equivalence Between Sparse Approximation and Support Vector Machines.

This article shows a relationship between two different approximation techniques: the support vector machines (SVM), proposed by V. Vapnik (1995) and a sparse approximation scheme that resembles the basis pursuit denoising algorithm (Chen, 1995; Chen, Donoho, and Saunders, 1995). SVM is a technique that can be derived from the structural risk minimization principle (Vapnik, 1982) and can be used to estimate the parameters of several different approximation schemes, including radial basis functions, algebraic and trigonometric polynomials, B-splines, and some forms of multilayer perceptrons. Basis pursuit denoising is a sparse approximation technique in which a function is reconstructed by using a small number of basis functions chosen from a large set (the dictionary). We show that if the data are noiseless, the modified version of basis pursuit denoising proposed in this article is equivalent to SVM in the following sense: if applied to the same data set, the two techniques give the same solution, which is obtained by solving the same quadratic programming problem. In the appendix, we present a derivation of the SVM technique in one framework of regularization theory, rather than statistical learning theory, establishing a connection between SVM, sparse approximation, and regularization theory.

Journal Article↗

PubMed1996

Predictive minimum description length criterion for time series modeling with neural networks.

Nonlinear time series modeling with a multilayer perceptron network is presented. An important aspect of this modeling is the model selection, i.e., the problem of determining the size as well as the complexity of the model. To overcome this problem we apply the predictive minimum description length (PMDL) principle as a minimization criterion. In the neural network scheme it means minimizing the number of input and hidden units. Three time series modeling experiments are used to examine the usefulness of the PMDL model selection scheme. A comparison with the widely used cross-validation technique is also presented. In our experiments the PMDL scheme and the cross-validation scheme yield similar results in terms of model complexity. However, the PMDL method was found to be two times faster to compute. This is significant improvement since model selection in general is very time consuming.

Algorithms↗