Nonlinear autoassociation is not equivalent to PCA.

N Japkowicz; Jos; M A Gluck

doi:10.1162/089976600300015691

Home/Archive/PubMed

PubMed · 10769321

Nonlinear autoassociation is not equivalent to PCA.

N Japkowicz·Jos·M A Gluck

View original source ↗Export metadata ↓

Abstract

A common misperception within the neural network community is that even with nonlinearities in their hidden layer, autoassociators trained with backpropagation are equivalent to linear methods such as principal component analysis (PCA). Our purpose is to demonstrate that nonlinear autoassociators actually behave differently from linear methods and that they can outperform these methods when used for latent extraction, projection, and classification. While linear autoassociators emulate PCA, and thus exhibit a flat or unimodal reconstruction error surface, autoassociators with nonlinearities in their hidden layer learn domains by building error reconstruction surfaces that, depending on the task, contain multiple local valleys. This interpolation bias allows nonlinear autoassociators to represent appropriate classifications of nonlinear multimodal domains, in contrast to linear autoassociators, which are inappropriate for such tasks. In fact, autoassociators with hidden unit nonlinearities can be shown to perform nonlinear classification and nonlinear recognition.

Explore related subjects

Linear Models Neural Networks, Computer Nonlinear Dynamics Normal Distribution Reproducibility of Results Journal Article Research Support, Non-U.S. Gov't

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

N Japkowicz, Jos, M A Gluck. 2000. Nonlinear autoassociation is not equivalent to PCA.. https://doi.org/10.1162/089976600300015691

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2025

A robust transfer learning approach for high-dimensional linear regression to support integration of multi-source gene expression data.

Transfer learning aims to integrate useful information from multi-source datasets to improve the learning performance of target data. This can be effectively applied in genomics when we learn the gene associations in a target tissue, and data from other tissues can be integrated. However, heavy-tail distribution and outliers are common in genomics data, which poses challenges to the effectiveness of current transfer learning approaches. In this paper, we study the transfer learning problem under high-dimensional linear models with t-distributed error (Trans-PtLR), which aims to improve the estimation and prediction of target data by borrowing information from useful source data and offering robustness to accommodate complex data with heavy tails and outliers. In the oracle case with known transferable source datasets, a transfer learning algorithm based on penalized maximum likelihood and expectation-maximization algorithm is established. To avoid including non-informative sources, we propose to select the transferable sources based on cross-validation. Extensive simulation experiments as well as an application demonstrate that Trans-PtLR demonstrates robustness and better performance of estimation and prediction when heavy-tail and outliers exist compared to transfer learning for linear regression model with normal error distribution. Data integration, Variable selection, T distribution, Expectation maximization algorithm, Genotype-Tissue Expression, Cross validation.

Linear Models↗

PubMed2007

A novel QSPR model for predicting θ (lower critical solution temperature) in polymer solutions using molecular descriptors.

In this study, we present a new model that has been developed for the prediction of θ (lower critical solution temperature) using a database of 169 data points that include 12 polymers and 67 solvents. For the characterization of polymer and solvent molecules, a number of molecular descriptors (topological, physicochemical,steric and electronic) were examined. The best subset of descriptors was selected using the elimination selection-stepwise regression method. Multiple linear regression (MLR) served as the statistical tool to explore the potential correlation among the molecular descriptors and the experimental data. The prediction accuracy of the MLR model was tested using the leave-one-out cross validation procedure, validation through an external test set and the Y-randomization evaluation technique. The domain of applicability was finally determined to identify the reliable predictions.

Linear Models↗

PubMed2006

Fit of four curve-linear models to decay profiles for pest control substances in soil.

Experiments that investigate the pattern of degradation of pest control substances in soil are often undertaken to estimate the persistence of compounds in the environment. Mathematical models are typically fit to decay data to facilitate the interpretation of the results and make predictions concerning the environmental fate of xenobiotics in soil. Four mathematical models were fit to 61 data sets to compare their performance in conforming to empirical patterns of degradation of pest control substances in soil. The use of composite residual plots allowed comparisons of the performance of the different models over many data sets. While an exponential model, estimated using nonlinear regression, fit many data sets very well, a shift-log, biexponential, and Monod equation appears superior in many cases, and systematic deviations from data sets are often less evident with the latter models. A knowledge of the patterns of bias typically exhibited by each model across many data sets may be useful for selecting models with reduced bias when fitting individual data sets.

Linear Models↗