SEARCH · PubMed Health

Results for “model selection”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 recordsLinked to original sources

PubMed2000

The Importance of Complexity in Model Selection.

Model selection should be based not solely on goodness-of-fit, but must also consider model complexity. While the goal of mathematical modeling in cognitive psychology is to select one model from a set of competing models that best captures the underlying mental process, choosing the model that best fits a particular set of data will not achieve this goal. This is because a highly complex model can provide a good fit without necessarily bearing any interpretable relationship with the underlying process. It is shown that model selection based solely on the fit to observed data will result in the choice of an unnecessarily complex model that overfits the data, and thus generalizes poorly. The effect of over-fitting must be properly offset by model selection methods. An application example of selection methods using artificial data is also presented. Copyright 2000 Academic Press.

Journal Article↗

PubMed2004

Model selection and model averaging in phylogenetics: advantages of akaike information criterion and bayesian approaches over likelihood ratio tests.

Model selection is a topic of special relevance in molecular phylogenetics that affects many, if not all, stages of phylogenetic inference. Here we discuss some fundamental concepts and techniques of model selection in the context of phylogenetics. We start by reviewing different aspects of the selection of substitution models in phylogenetics from a theoretical, philosophical and practical point of view, and summarize this comparison in table format. We argue that the most commonly implemented model selection approach, the hierarchical likelihood ratio test, is not the optimal strategy for model selection in phylogenetics, and that approaches like the Akaike Information Criterion (AIC) and Bayesian methods offer important advantages. In particular, the latter two methods are able to simultaneously compare multiple nested or nonnested models, assess model selection uncertainty, and allow for the estimation of phylogenies and model parameters using all available models (model-averaged inference or multimodel inference). We also describe how the relative importance of the different parameters included in substitution models can be depicted. To illustrate some of these points, we have applied AIC-based model averaging to 37 mitochondrial DNA sequences from the subgenus Ohomopterus(genus Carabus) ground beetles described by Sota and Vogler (2001).

Animals↗

PubMed2003

On the undecidability among kinetic models: from model selection to model averaging.

This article deals with the problem of model selection for the mathematical description of tracer kinetics in nuclear medicine. It stems from the consideration of some specific data sets where different models have similar performances. In these situations, it is shown that considerate averaging of a parameter's estimates over the entire model set is better than obtaining the estimates from one model only. Furthermore, it is also shown that the procedure of averaging over a small number of "good" models reduces the "generalization error," the error introduced when the model selected over a particular data set is applied to different conditions, such as subject populations with altered physiologic parameters, modified acquisition protocols, and different signal-to-noise ratios. The method of averaging over the entire model set uses Akaike coefficients as measures of an individual model's likelihood. To facilitate the understanding of these statistical tools, the authors provide an introduction to model selection criteria and a short technical treatment of Akaike's information-theoretic approach. The new method is illustrated and epitomized by a case example on the modeling of [11C]flumazenil kinetics in the brain, containing both real and simulated data.

Carbon Radioisotopes↗

PubMed2000

Bayesian Model Selection and Model Averaging.

This paper reviews the Bayesian approach to model selection and model averaging. In this review, I emphasize objective Bayesian methods based on noninformative priors. I will also discuss implementation details, approximations, and relationships to other methods. Copyright 2000 Academic Press.

Journal Article↗

PubMed2003

A sensibility analysis of model selection in modeling the reactive transport of cesium in crushed granite.

We performed a sensibility analysis of model selection in modeling the reactive transport of cesium in crushed granite through model calibration and validation. Based on some solid phase analysis data and kinetic batch experimental results, we hypothesized three two-site sorption models in the LEHGC reactive transport model to fit the breakthrough curves (BTCs) from the corresponding column experiments. The analysis of breakthrough curves shows that both the empirical two-site kinetic linear sorption model and the semi-mechanistic/semi-empirical two-site kinetic surface complexation model, regardless of their complexity, can match our experimental data fairly well under given test conditions. A numerical experiment to further compare the two models shows that they behave differently when the pore velocity is not of the same order of magnitude as our test velocities. This result indicates that further investigations to help determine a better model are needed. We suggest that a multistage column experiment, which tests over the whole range of practical flow velocities, should be conducted to help alleviate inadequate hypothesized models.

Calibration↗

PubMed2005

Germline bottlenecks, biparental inheritance and selection on mitochondrial variants: a two-level selection model.

Selection on mitochondrial mutations potentially occurs at different levels: at the mitochondria, cell, and organism levels. Several factors affect the strength of selection at these different levels; in particular, mitochondrial bottlenecks during germline development and reduced paternal transmission decrease the genetic variance within cells, while they increase the variance between cells and between organisms, thus decreasing the strength of selection within cells and increasing the strength of selection between cells and organisms. However, bottlenecks and paternal transmission also affect the effective mitochondrial population size, thus affecting genetic drift. In this article, we use a simple model of a unicellular life cycle to investigate the effects of bottlenecks and paternal transmission on the probability of fixation of mitochondrial mutants and their frequency at mutation-selection equilibrium. We find that bottlenecks and reduced paternal transmission decrease the mean frequency of alleles with sm>sc (approximately), where sm and sc are the strengths of selection for an allele within and between cells, respectively, and increase the frequency of alleles with sm 0 (unless sm is very small relative to sc) and increase the fixation probability of mutants with sm<0.

DNA, Mitochondrial↗

PubMed2006

Model selection for geostatistical models.

We consider the problem of model selection for geospatial data. Spatial correlation is often ignored in the selection of explanatory variables, and this can influence model selection results. For example, the importance of particular explanatory variables may not be apparent when spatial correlation is ignored. To address this problem, we consider the Akaike Information Criterion (AIC) as applied to a geostatistical model. We offer a heuristic derivation of the AIC in this context and provide simulation results that show that using AIC for a geostatistical model is superior to the often-used traditional approach of ignoring spatial correlation in the selection of explanatory variables. These ideas are further demonstrated via a model for lizard abundance. We also apply the principle of minimum description length (MDL) to variable selection for the geostatistical model. The effect of sampling design on the selection of explanatory covariates is also explored. R software to implement the geostatistical model selection methods described in this paper is available in the Supplement.

Animals↗

PubMed1997

Validity of methods for model selection, weighting for model uncertainty, and small sample adjustment in capture-recapture estimation.

In log-linear capture-recapture approaches to population size, the method of model selection may have a major effect upon the estimate. In addition, the estimate may also be very sensitive if certain cells are null or very sparse, even with the use of multiple sources. The authors evaluated 1) various approaches to the issue of model uncertainty and 2) a small sample correction for three or more sources recently proposed by Hook and Regal. The authors compared the estimates derived using 1) three different information criteria that included Akaike's Information Criterion (AIC) and two alternative formulations of the Bayesian Information Criterion (BIC), one proposed by Draper ("two pi") and one by Schwarz ("not two pi"); 2) two related methods of weighting estimates associated with models; 3) the independent model; and 4) the saturated model, with the known totals in 20 different populations studied by five separate groups of investigators. For each method, we also compared the estimate derived with or without the proposed small sample correction. At least in these data sets, the use of AIC appeared on balance to be preferable. The BIC formulation suggested by Draper appeared slightly preferable to that suggested by Schwarz. Adjustment for model uncertainty appears to improve results slightly. The proposed small sample correction appeared to diminish relative log bias but only when sparse cells were present. Otherwise, its use tended to increase relative log bias. Use of the saturated model (with or without the small sample correction) appears to be optimal if the associated interval is not uselessly large, and if one can plausibly exclude an all-source interaction. All other approaches led to an estimate that was too low by about one standard deviation.

Bayes Theorem↗

PubMed2003

The use of model selection in the model-free analysis of protein dynamics.

Model-free analysis of NMR relaxation data, which is widely used for the study of protein dynamics, consists of the separation of the global rotational diffusion from internal motions relative to the diffusion frame and the description of these internal motions by amplitude and timescale. Five model-free models exist, each of which describes a different type of motion. Model-free analysis requires the selection of the model which best describes the dynamics of the NH bond. It will be demonstrated that the model selection technique currently used has two significant flaws, under-fitting, and not selecting a model when one ought to be selected. Under-fitting breaks the principle of parsimony causing bias in the final model-free results, visible as an overestimation of S2 and an underestimation of taue and Rex. As a consequence the protein falsely appears to be more rigid than it actually is. Model selection has been extensively developed in other fields. The techniques known as Akaike's Information Criteria (AIC), small sample size corrected AIC (AICc), Bayesian Information Criteria (BIC), bootstrap methods, and cross-validation will be compared to the currently used technique. To analyse the variety of techniques, synthetic noisy data covering all model-free motions was created. The data consists of two types of three-dimensional grid, the Rex grids covering single motions with chemical exchange [S2,taue,Rex], and the Double Motion grids covering two internal motions [S f 2,S s 2,tau s ]. The conclusion of the comparison is that for accurate model-free results, AIC model selection is essential. As the method neither under, nor over-fits, AIC is the best tool for applying Occam's razor and has the additional benefits of simplifying and speeding up model-free analysis.

Amines↗

PubMed2000

Intrinsic priors for model selection using an encompassing model with applications to censored failure time data.

In Bayesian model selection or testing problems one cannot utilize standard or default noninformative priors, since these priors are typically improper and are defined only up to arbitrary constants. Therefore, Bayes factors and posterior probabilities are not well defined under these noninformative priors, making Bayesian model selection and testing problems impossible. We derive the intrinsic Bayes factor (IBF) of Berger and Pericchi (1996a, 1996b) for the commonly used models in reliability and survival analysis using an encompassing model. We also derive proper intrinsic priors for these models, whose Bayes factors are asymptotically equivalent to the respective IBFs. We demonstrate our results in three examples.

Bayes Theorem↗

PubMed1994

In favor of the selective model of positive selection.

The mechanisms of thymocyte commitment towards the CD4+ and CD8+ lineage remain unresolved. Two models--one based on instruction, the other on selection--have previously been proposed. The instructional model has been popularly received based on results of earlier studies. However, our data from MHC class II, class I, and double-deficient mice suggest otherwise. There exists a significant population of CD4+ cells that is intermediate in maturity between CD4+ CD8+ and fully mature CD4+ CD8- thymocytes in class II-deficient animals; an analogous population of CD4-CD8+ cells exists in class I-negative mice. We suggest that a selective model in which two TCR-MHC molecule engagements are required: the first induces a random down-modulation of either CD4 or CD8 and some differentiation; the second, involving the participation of the appropriate coreceptor, permits end-stage differentiation.

Animals↗

PubMed1993

Another view of the selective model of thymocyte selection.

Thymocyte commitment to the CD4 helper versus CD8 cytotoxic lineage has not been satisfactorily established. Two models have been elaborated: one based on instruction, the other on selection. Most previous results support the instructive model, but our comparison of thymocyte differentiation in MHC class II-, class I- and double-deficient mice provides data challenging it. There exists a significant population of CD4 single positive cells in class II-deficient animals that is intermediate in maturity between CD4+CD8+ and end-stage CD4+CD8- thymocytes and is selected on class I molecules; an equivalent CD8+CD4- population occurs in class I-deficient animals. We propose a selective model entailing two TCR-MHC molecule engagements: the first provokes random down-modulation of either CD4 or CD8 and a degree of differentiation; the second, requiring participation of the appropriate coreceptor, permits end-stage differentiation.

Animals↗

PubMed2006

Model selection and mixed-effects modeling of HIV infection dynamics.

We present an introduction to a model selection methodology and an application to mathematical models of in vivo HIV infection dynamics. We consider six previously published deterministic models and compare them with respect to their ability to represent HIV-infected patients undergoing reverse transcriptase mono-therapy. In the creation of the statistical model, a hierarchical mixed-effects modeling approach is employed to characterize the inter- and intra-individual variability in the patient population. We estimate the population parameters in a maximum likelihood function formulation, which is then used to calculate information theory based model selection criteria, providing a ranking of the abilities of the various models to represent patient data. The parameter fits generated by these models, furthermore, provide statistical support for the higher viral clearance rate c in Louie et al. [AIDS 17:1151-1156, 2003]. Among the candidate models, our results suggest which mathematical structures, e.g., linear versus nonlinear, best describe the data we are modeling and illustrate a framework for others to consider when modeling infectious diseases.

CD4 Lymphocyte Count↗

PubMed2006

The time-profile of cell growth in fission yeast: model selection criteria favoring bilinear models over exponential ones.

BACKGROUND: There is considerable controversy concerning the exact growth profile of size parameters during the cell cycle. Linear, exponential and bilinear models are commonly considered, and the same model may not apply for all species. Selection of the most adequate model to describe a given data-set requires the use of quantitative model selection criteria, such as the partial (sequential) F-test, the Akaike information criterion and the Schwarz Bayesian information criterion, which are suitable for comparing differently parameterized models in terms of the quality and robustness of the fit but have not yet been used in cell growth-profile studies. RESULTS: Length increase data from representative individual fission yeast (Schizosaccharomyces pombe) cells measured on time-lapse films have been reanalyzed using these model selection criteria. To fit the data, an extended version of a recently introduced linearized biexponential (LinBiExp) model was developed, which makes possible a smooth, continuously differentiable transition between two linear segments and, hence, allows fully parametrized bilinear fittings. Despite relatively small differences, essentially all the quantitative selection criteria considered here indicated that the bilinear model was somewhat more adequate than the exponential model for fitting these fission yeast data. CONCLUSION: A general quantitative framework was introduced to judge the adequacy of bilinear versus exponential models in the description of growth time-profiles. For single cell growth, because of the relatively limited data-range, the statistical evidence is not strong enough to favor one model clearly over the other and to settle the bilinear versus exponential dispute. Nevertheless, for the present individual cell growth data for fission yeast, the bilinear model seems more adequate according to all metrics, especially in the case of wee1Delta cells.

Biophysics↗

PubMed2006

Bayesian feature and model selection for Gaussian mixture models.

We present a Bayesian method for mixture model training that simultaneously treats the feature selection and the model selection problem. The method is based on the integration of a mixture model formulation that takes into account the saliency of the features and a Bayesian approach to mixture learning that can be used to estimate the number of mixture components. The proposed learning algorithm follows the variational framework and can simultaneously optimize over the number of components, the saliency of the features, and the parameters of the mixture model. Experimental results using high-dimensional artificial and real data illustrate the effectiveness of the method.

Algorithms↗

PubMed2000

Key Concepts in Model Selection: Performance and Generalizability.

What is model selection? What are the goals of model selection? What are the methods of model selection and how do they work? Which methods perform better than others and in what circumstances? These questions rest on a number of key concepts in a relatively underdeveloped field. The aim of this paper is to explain some background concepts, to highlight some of the results in this special issue, and to add my own. The standard methods of model selection include classical hypothesis testing, maximum likelihood, Bayes method, minimum description length, cross-validation, and Akaike's information criterion. They all provide an implementation of Occam's razor, in which parsimony or simplicity is balanced against goodness-of-fit. These methods primarily take account of the sampling errors in parameter estimation, although their relative success at this task depends on the circumstances. However, the aim of model selection should also include the ability of a model to generalize to predictions in a different domain. Errors of extrapolation, or generalization, are different from errors of parameter estimation. So, it seems that simplicity and parsimony may be an additional factor in managing these errors, in which case the standard methods of model selection are incomplete implementations of Occam's razor. Copyright 2000 Academic Press.

Journal Article↗

PubMed2001

Model selection in estimating equations.

Model selection is a necessary step in many practical regression analyses. But for methods based on estimating equations, such as the quasi-likelihood and generalized estimating equation (GEE) approaches, there seem to be few well-studied model selection techniques. In this article, we propose a new model selection criterion that minimizes the expected predictive bias (EPB) of estimating equations. A bootstrap smoothed cross-validation (BCV) estimate of EPB is presented and its performance is assessed via simulation for overdispersed generalized linear models. For illustration, the method is applied to a real data set taken from a study of the development of ewe embryos.

Bias↗

PubMed2007

Estimating the predictive quality of dose-response after model selection.

Prediction of dose-response is important in dose selection in drug development. As the true dose-response shape is generally unknown, model selection is frequently used, and predictions based on the final selected model. Correctly assessing the quality of the predictions requires accounting for the uncertainties caused by the model selection process, which has been difficult. Recently, a new approach called data perturbation has emerged. It allows important predictive characteristics be computed while taking model selection into consideration. We study, through simulation, the performance of data perturbation in estimating standard error of parameter estimates and prediction errors. Data perturbation was found to give excellent prediction error estimates, although at times large Monte Carlo sizes were needed to obtain good standard error estimates. Overall, it is a useful tool to characterize uncertainties in dose-response predictions, with the potential of allowing more accurate dose selection in drug development. We also look at the influence of model selection on estimation bias. This leads to insights into candidate model choices that enable good dose-response prediction.

Bias↗