On cross validation for model selection.

I Rivals; L Personnaz

doi:10.1162/089976699300016476

Home/Archive/PubMed

PubMed · 10226186

On cross validation for model selection.

I Rivals·L Personnaz

View original source ↗Export metadata ↓

Abstract

In response to Zhu and Rower (1996), a recent communication (Goutte, 1997) established that leave-one-out cross validation is not subject to the "no-free-lunch" criticism. Despite this optimistic conclusion, we show here that cross validation has very poor performances for the selection of linear models as compared to classic statistical tests. We conclude that the statistical tests are preferable to cross validation for linear as well as for nonlinear model selection.

Explore related subjects

Linear Models Models, Statistical Nonlinear Dynamics Probability Reproducibility of Results Journal Article

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

I Rivals, L Personnaz. 1999-05-15. On cross validation for model selection.. https://doi.org/10.1162/089976699300016476

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2025

A robust transfer learning approach for high-dimensional linear regression to support integration of multi-source gene expression data.

Transfer learning aims to integrate useful information from multi-source datasets to improve the learning performance of target data. This can be effectively applied in genomics when we learn the gene associations in a target tissue, and data from other tissues can be integrated. However, heavy-tail distribution and outliers are common in genomics data, which poses challenges to the effectiveness of current transfer learning approaches. In this paper, we study the transfer learning problem under high-dimensional linear models with t-distributed error (Trans-PtLR), which aims to improve the estimation and prediction of target data by borrowing information from useful source data and offering robustness to accommodate complex data with heavy tails and outliers. In the oracle case with known transferable source datasets, a transfer learning algorithm based on penalized maximum likelihood and expectation-maximization algorithm is established. To avoid including non-informative sources, we propose to select the transferable sources based on cross-validation. Extensive simulation experiments as well as an application demonstrate that Trans-PtLR demonstrates robustness and better performance of estimation and prediction when heavy-tail and outliers exist compared to transfer learning for linear regression model with normal error distribution. Data integration, Variable selection, T distribution, Expectation maximization algorithm, Genotype-Tissue Expression, Cross validation.

Linear Models↗

PubMed1999

Graph-theoretic description of the interplay between non-linearity and connectivity in biological systems.

The purpose of this article is to stress the implications that the consideration of nonlinearity has upon the extension and strength of connectivity, if this is understood as a characterization of the degree of interrelation between parts of the system. This objective is reached within the QP formalism for non-linear ODEs. The formalism is developed in a graph-theoretic setting, with the help of which the connectionist aspect of non-linearity becomes apparent. Topology-preserving transformations involve an exchange between the degree of non-linearity and the strengths of interactions, thus assembling systems of apparently different nature into classes of equivalence. We argue that, if we have in mind a classification of systems according to behavior, these classes of equivalence should be given their proper singularity. We characterize globally the connectivity of a class with an index, although we point out during the discussion that the mathematical conception of the complex idea of connectivity is still incomplete.

Linear Models↗

PubMed1999

Attenuation of cross-peak intensities in QUIET-BIRD-NOESY experiments.

The buildup curves in QUIET-BIRD-NOESY experiments, which are designed to isolate two-spin subsystems within macromolecules, are attenuated by transverse relaxation and evolution under homonuclear couplings during the bilinear rotation decoupling (BIRD) pulse sandwich. If the signals of both source and target spins are attenuated equally (uniform damping), this is readily accounted for by normalizing the cross peaks with respect to the diagonal peaks. However, unequal attenuation of source and target spins (differential damping) affects the initial buildup slopes and hence leads to apparent cross-relaxation rates that are significantly distorted from their true values. A simple method for recognizing this situation and extracting accurate cross-relaxation rates is presented.

Linear Models↗