SEARCH · PubMed Health

Results for “external validation”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5Linked to original sources

PubMed1990

The impact of methodological factors on child psychotherapy outcome research: a meta-analysis for researchers.

Two recent meta-analyses have generated evidence for child and adolescent psychotherapy effects. However, critics note that such meta-analyses often include studies with methodological shortcomings which might invalidate their results. In the present study, we explored whether the results of the most extensive child/adolescent meta-analysis might have been influenced by such methodological variables, focusing on internal validity and external validity factors. Together, these factors accounted for two-thirds as much variance as the substantive factors (e.g., type of therapy, age) in the original meta-analysis. This suggests that relative to these therapy and child-characteristic variables, methodological factors have a substantial, though smaller, impact on meta-analysis results. In general, increased experimental rigor was related to larger effect sizes; this argues against the hypothesis that methodologically weak studies have led to an overestimate of therapy effects. No significant interactive relations were found between validity factors and predictors of outcome; this suggests that the relations noted in previous meta-analyses between outcome and various variables were not distorted by the validity factors tested here.

Adaptation, Psychological↗

PubMedDate not supplied

The use of claims databases for outcomes research: rationale, challenges, and strategies.

Health care payers and policy makers need information about the cost and effectiveness of medical treatments. While randomized controlled trials historically are the primary source of medical information, they are expensive and labor-intensive, and often have limited utility for answering questions about "real-world" patient populations. These problems have led to an increasing reliance on claims database research in making policy decisions about treatment options. However, both researchers and decision makers should recognize the limitations and unique features of claims databases. Recommendations for avoiding or minimizing threats to internal validity, construct validity, and external validity are: (1) use of a study design that includes comparisons; (2) ensuring that the study design and conclusions are consistent with the database; (3) a priori conceptual modeling of the research question; (4) use of appropriate constructs; (5) explicit examination of alternative explanations for study findings; (6) sensitivity analyses of key assumptions; (7) awareness of the distinction between statistical and practical significance of findings; (8) generalization only when appropriate; and (9) reporting of relevant information. Given that any study design or data source has limitations, we hope that this paper will encourage a philosophy of methodological pluralism in outcomes research. Awareness and accurate reporting of validity issues will strengthen and extend the information resources currently available to decision makers.

Decision Making↗

PubMed1997

Reliability of comparative molecular field analysis models: effects of data scaling and variable selection using a set of human synovial fluid phospholipase A2 inhibitors.

The effects of data pretreatment, data scaling, and variable selection on three-dimensional quantitative structure-activity relationships derived by comparative molecular field analysis (CoMFA) using the GRID energy function were studied in detail for a set of inhibitors of the human synovial fluid phospholipase A2 (HSF-PLA2). The quality of the models was evaluated for predictive power and ability to map the receptor binding site by (a) comparison of predicted and experimental activities using cross-validation and external validation sets and (b) comparison of the regions selected in space in the CoMFA models with a crystal structure of a HSF-PLA2-inhibitor complex, with optimized comparative binding energy analysis (COMBINE) models (Ortiz et al., 1995) and with structure-activity relationships derived previously for different sets of compounds. It is found that (1) data scaling and dielectric modeling strongly influence CoMFA results. Unscaled data and a uniform dielectric constant of 4 are well suited to GRID-CoMFA studies for the present compound set. (2) The GOLPE and Q2-GRS variable selection methods select variables in roughly the same regions in Cartesian space, but they produce different models in chemometric space and differ in their sensitivity to data scaling and pretreatment and their tendency to overfitting. (3) CoMFA models are consistent with COMBINE models in that they identify approximately the same intermolecular interactions as relevant for activity. Our study provides support for the qualitative receptor-mapping properties of CoMFA models and for the validity of variable selection when applied with care and also provides guidelines for how to evaluate the quality of CoMFA models.

Enzyme Inhibitors↗

PubMed1994

Prediction equations for the estimation of body composition in the elderly using anthropometric data.

To study the relationship between health and nutritional status in elderly populations, information about body composition is essential. To collect this information in large epidemiological studies, practical methods based on anthropometric data must be available. In the present study the relationship between body composition, determined by densitometry, and anthropometric data in 204 elderly men and women, aged 60-87 years, was analysed. Existing prediction equations described in the literature, and mainly based on young and middle-aged subjects, generally underestimated percentage body fat in the elderly study population. Therefore, new prediction equations were developed, based on sex and the sum of two (biceps and triceps) or four (biceps, triceps, suprailiaca and subscapula) skinfolds or the body mass index (BMI). Addition of age or body circumferences to the models did not improve the prediction of body density. Internal cross validation and external validation revealed that the formulas are valid for the estimation of body density in elderly subjects. The standard errors of estimate of the three models, expressed as percentage body fat, were 5.6, 5.4 and 4.8% respectively.

Aged↗

PubMed1992

The structure of the Mental Health Inventory among Chinese in Taiwan.

This study attempted to ascertain the construct validity and external validity of the Mental Health Inventory in a Chinese population in Taiwan and contrast these results with results obtained from studies of several U.S. populations. In particular, a series of measurement models were specified and evaluated to address the issues of reliability and validity. Data were collected from personal interviews of a probability sample of 1,194 Chinese respondents 14 years of age and older in four townships in southwest Taiwan. The Mental Health Inventory was found to involve two major components: positive well-being and psychological distress. As a hierarchical structure, each component consists of one second-order and two or three first-order factors. The relationships between well-being and distress can be characterized as substantially independent and modestly bipolar depending on the level and specification.

Adolescent↗

PubMed1996

Using vignettes to collect data for nursing research studies: how valid are the findings?

Vignettes are simulations of real events which can be used in research studies to elicit subject's knowledge, attitudes or opinions according to how they state they would behave in the hypothetical situation depicted. Advantages associated with the use of vignettes as research tools include: the ability to collect information simultaneously from large numbers of subjects, to manipulate a number of variables at once in a manner that would not be possible in observation studies, absence of observer effect and avoidance of the ethical dilemmas commonly encountered during observation. Difficulties include problems establishing reliability and validity, especially external validity. This paper considers the advantages and disadvantages associated with the use of vignettes as data collection tools, concluding with a check-list to help critique vignettes studies.

Bias↗

PubMed1992

Development of an interview-based geriatric depression rating scale.

The geriatric depression rating scale (GDRS) is a new interview-based depression rating scale designed for use with adults 60 years of age or older. The scale was developed to fill a need for an instrument that would be sensitive to the problems encountered in assessing depression among older adults. The GDRS was designed by using items from the self-report Geriatric Depression Scale (GDS) as topic areas in a structured clinical interview similar to that of the Hamilton Rating Scale for Depression (HRSD). The 35-item rating scale was administered to 68 older individuals with a range of affective disturbance. The scale was found to have internal consistency and split-half reliability comparable to the HRSD and GDS. Concurrent validity, construct validity, external criterion validity, sensitivity, and specificity were all found to be acceptable.

Aged↗

PubMed1978

A critique of project evaluations.

In recent years an increased stress has been placed on the evaluation of mental health, education, and welfare service programs. The majority of studies readily available to most evaluators represent local project evaluations which usually contain diverse references to different aspects of the evaluative process. For evaluative results to be even minimally useful to other projects, however, certain requirements must be met. These are: (1) internal validity, (2) external validity, (3) specification of the population and treatment being implemented, and (4) standardization of indicators of treatment impact. To determine the extent to which published project impact evaluations meet these criteria, a study was undertaken to "evaluate the evaluations" themselves within heroin addiction treatment programs. Six high-yield journals and 100 random sources were systematically searched for reports of evaluations which provided measures of success in terms of the consumer. Articles were analyzed in regard to our four prerequisites for cross-project comparisons regarding process variables, impact variables, and methodologies. It became clear, however, that our original objectives in evaluating either the usefulness of published project evaluations or testing any specific impact hypotheses were not achievable due to the state of evaluative measurement and reporting practices at this time. The major problems we eoncountered in our inability to complete a necessary and potentially fruitful comparative assessment of project evaluations are discussed in detail with recommendations for future work.

Heroin Dependence↗

PubMed2026

Analysis of end-stage renal disease mediated by cuproptosis-related genes.

OBJECTIVE: The complex pathophysiological mechanism of end-stage renal disease (ESRD) has not been fully understood. Cuproptosis is a newly discovered type of programmed cell death. Therefore, this study attempts to clarify the relationship between cuproptosis-related genes (CRGs) and the phenotype of ESRD. MATERIALS AND METHODS: The National Center for Biological Information Gene Expression Omnibus database was applied to obtain the GSE37171 dataset comprising whole-genome microarray analysis of peripheral blood samples. A 3 : 1 case-control design was employed with 75 ESRD patients and 20 healthy controls who were frequency-matched for age, sex, and ethnicity. Based on differentially expressed genes (DEGs) and genes related to cuproptosis, CRGs were identified. Thereafter, we explored two different subpopulations based on the cuproptosis gene and analyzed their expression and immune infiltration. Genes specific to the CRG cluster were identified through the weighted gene co-expression network analysis algorithm, and the best prediction model was determined and verified by four machine learning methods. RESULTS: The study identified 14 differentially expressed CRGs, among which ATP7B, SLC31A1, LIAS, LIPT1, DLD, MTF1, CDKN2A, DBT, and DLST had relatively high expression levels in the ESRD samples. Compared with the control group, expression levels of FDX1, DLAT, PDHA1, PDHB, and GLS were significantly lower in the ESRD group, and CRGs played a key role in the regulation of immune infiltration in ESRD. Two cuproptosis-related molecular clusters were identified in the ESRD samples. Cluster2 was more correlated with the immune infiltration of ESRD. By analyzing the intersection points between CRG cluster and key genes of ESRD, a total of 888 specific DEGs were identified. Functional differences related to specific DEGs were further explored using gene set variation analysis. Five significant genes (SMC5, USP47, USP53, AGA, and DMXL1) were identified by the support vector machine model as key predictors for ESRD disease risk, achieving an area under the curve (AUC) of 1.00 in internal validation. However, external validation in independent cohorts is required prior to clinical application. Individual gene analysis showed an AUC > 0.81 in discriminating ESRD patients from healthy controls, and the expression of all 5 genes in ESRD patients was significantly lower than in the control group. CONCLUSION: This study clarified the relationship between CRGs and the phenotype of ESRD, analyzed their specific roles in the immune microenvironment, and obtained a predictive model, providing new insights for the study of its potential therapeutic targets.

Humans↗

PubMed1990

A methodological framework for the design of research on the evaluation of residents.

This paper describes a construct validation framework for research on the selection and evaluation of residents. The application of the proposed methodology to surgery residents is described. The need to measure non-cognitive and neuropsychological factors in addition to cognitive knowledge and technical ability is emphasized, and a research strategy that integrates theory formulation, internal validation, and external validation is presented. In this context, residents' competence is viewed as a multivariate construct that requires validation through longitudinal empirical studies and the use of multivariate statistical approaches.

Clinical Competence↗

PubMed1984

Scientific challenges in the application of randomized trials.

In recent years, scientific challenges in the application of randomized trials have become more apparent, especially with the extension of such trials to the assessment of nondrug treatments, such as health education, psychotherapy, and health care provision. Six issues (individual v group randomization, blinding and unblinding, the effect of trial participation on outcome, selective subject participation, treatment compliance, and standardized v individualized treatment) are discussed in terms of their impact on internal validity, generalizability (external validity), and clinical relevance. Specific design strategies may be necessary to enhance these methodological and clinical desiderata. Attention to these challenges should lead to improvements in future randomized trials.

Attitude↗

PubMed1999

Sherlock Holmes and child psychopathology assessment approaches: the case of the false-positive.

OBJECTIVE: To explore the relative value of various methods of assessing childhood psychopathology, the authors compared 4 groups of children: those who met criteria for one or more DSM diagnoses and scored high on parent symptom checklists, those who met psychopathology criteria on either one of these two assessment approaches alone, and those who met no psychopathology assessment criterion. METHOD: Parents of 201 children completed the Child Behavior Checklist (CBCL), after which children and parents were administered the Diagnostic Interview Schedule for Children (version 2.1). Children and parents also completed other survey measures and symptom report inventories. The 4 groups of children were compared against "external validators" to examine the merits of "false-positive" and "false-negative" cases. RESULTS: True-positive cases (those that met DSM criteria and scored high on the CBCL) differed significantly from the true-negative cases on most external validators. "False-positive" and "false-negative" cases had intermediate levels of most risk factors and external validators. "False-positive" cases were not normal per se because they scored significantly above the true-negative group on a number of risk factors and external validators. A similar but less marked pattern was noted for "false-negatives." CONCLUSIONS: Findings call into question whether cases with high symptom checklist scores despite no formal diagnoses should be considered "false-positive." Pending the availability of robust markers for mental illness, researchers and clinicians must resist the tendency to reify diagnostic categories or to engage in arcane debates about the superiority of one assessment approach over another.

Adaptation, Psychological↗

PubMed2026

Could the preoperative urethral curve be used to predict immediate urinary continence following Retzius-sparing robot-assisted radical prostatectomy? A retrospective multi-center study.

PURPOSE: Immediate urinary continence (UC) recovery following Retzius-sparing robot-assisted radical prostatectomy (RS-RARP) remains highly variable, highlighting the need for reliable preoperative prediction. We aimed to develop and validate models to identify patients likely to achieve immediate UC recovery following RS-RARP. MATERIALS AND METHODS: A total of 580 prostate cancer patients who underwent RS-RARP from four medical centers were assigned to a training set (n=348), an internal validation set (n=103) and an external validation set (n=129). Independent predictors were identified through univariate analysis and LASSO regression. A nomogram was constructed using multivariate logistic regression. Its performance was evaluated with receiver operating characteristic (ROC) curve, calibration curves, and decision curve analysis. RESULTS: Immediate UC recovery was observed in 84.5% (294/348) of patients in the training cohort, 80.6% (83/103) in the internal validation cohort, and 81.4% (105/129) in the external validation cohort, respectively. Multivariate analysis identified membranous urethral length (MUL) (OR=1.23, P=0.029) and urethral curvature (OR=2.84, P<0.001) as independent predictors, while prostate volume (PV) (OR=0.84, P <0.001) as a protective factor. The nomogram integrating MUL, PV, and urethral curvature demonstrated superior predictive accuracy, with an AUC of 0.87 (95% CI, 0.83-0.91) in the training cohort. The bootstrap-corrected calibration slope was 0.96, and the Brier score was 0.08. Calibration curves and decision curve analysis confirmed the predictive accuracy and clinical utility of the nomogram. CONCLUSIONS: Our study introduces a novel quantitative method for assessing urethral curvature. The mpMRI-based model, integrating urethral curvature and prostate spatial configuration, offers enhanced predictive accuracy for postoperative immediate UC recovery.

Humans↗

PubMed1997

Maxillary sinusitis in adults: an evaluation of placebo-controlled double-blind trials.

BACKGROUND: In general practice, acute sinusitis is frequently diagnosed and treated with antibiotics. OBJECTIVE: This study aimed to determine the evidence for the effectiveness of antibiotic treatment in acute maxillary sinusitis in adults by assessing the methodological quality of placebo-controlled double-blind randomized trials. METHOD: An evaluation by four raters through a 35-item scoring-scale for internal and external validity of all placebo-controlled double-blind randomized trials on acute sinusitis found between January 1966 and July 1996. RESULTS: Eighty-five trials were excluded because they were not placebo-controlled, double-blind, randomized, or were carried out in patients with chronic sinusitis or in children. The three remaining trials were performed in different populations (one in general practice) between 1973 and 1978. Only one study claimed superiority of antibiotic treatment. Different inclusion criteria and major outcome measures were used by the authors. The reliability of major outcome events was reported poorly or not at all and in two studies outcome measures were clinically inappropriate. The studies scored 30-62% of the maximum attainable score for internal validity and 10-20% for external validity. CONCLUSION: The effectiveness of antibiotic treatment in acute maxillary sinusitis in a general practice population is not based sufficiently on evidence.

Acute Disease↗

PubMed2026

Diagnostic performance of machine learning models versus established risk stratification for intracranial aneurysm rupture: a systematic review and bivariate meta-analysis.

BACKGROUND: Machine learning (ML) models have been proposed to improve the discrimination of intracranial aneurysm rupture status beyond established clinical risk stratification tools. However, reported performance is heterogeneous and the relative contribution of model architecture and feature dominance remains unclear. METHODS: We performed a Preferred Reporting Items for Systematic Reviews and Meta-Analyses-diagnostic test accuracy systematic review and diagnostic meta-analysis of studies evaluating ML models for intracranial aneurysm rupture discrimination. PubMed, Embase and CENTRAL were searched to February 2026. Sensitivity and specificity were pooled using a bivariate random-effects model, with summary receiver operating characteristic curves generated across training, internal testing and external validation datasets. Models were compared with regression-based approaches and Population, Hypertension, Age, Size of aneurysm, Earlier subarachnoid haemorrhage, Site of aneurysm (PHASES) scores. Subgroup and meta-regression analyses explored associations between algorithm family and feature domain. RESULTS: Sixty-two retrospective cohorts (29 709 patients 209 models) met the inclusion criteria. In training datasets, pooled sensitivity and specificity for ML were 0.81 (95% CI 0.75 to 0.85) and 0.83 (0.80-0.86), with an area under the curve (AUC) of 0.878, exceeding PHASES (AUC 0.667). In testing datasets, ML retained higher discrimination (AUC 0.837) than regression models (0.806) and PHASES (0.646). In external validation, sensitivity was preserved (0.82), but specificity declined (0.66). Deep learning demonstrated the highest AUCs (training and testing). Incorporation of haemodynamic or radiomic features improved pooled discrimination relative to morphology alone. Evidence of small-study effects and mostly unclear Prediction Model Risk Of Bias Assessment Tool ratings were observed. CONCLUSIONS: ML approaches demonstrate higher pooled discrimination for aneurysm rupture status than conventional risk scores in retrospective datasets, but reduced external validation specificity and heterogeneity limit confidence for clinical translation. Prospective, externally validated, calibrated models are required before integration into routine cerebrovascular risk stratification.

Humans↗

PubMed2026

Prediction of incident heart failure in established atherosclerotic cardiovascular disease: the SMART2-HF model.

BACKGROUND AND AIMS: Patients with established atherosclerotic cardiovascular disease (ASCVD) are at high risk of developing heart failure (HF). However, incident HF is not part of the risk assessment of current guideline-recommended models. The aim of this study was to develop and externally validate the SMART2-HF model for prediction of incident HF in patients with ASCVD. METHODS: SMART2-HF was developed in 7698 individuals with established ASCVD (coronary, cerebrovascular, or peripheral artery disease, or abdominal aortic aneurysm) but without prior HF from the UCC-SMART cohort. Cox proportional hazards models including sex-predictor interactions and with age as the time scale were derived to estimate the 10-year and lifetime risk of incident HF (hospitalization for HF or HF-related death), accounting for competing non-HF mortality. Predictors, limited to routinely available clinical characteristics, were aligned with the SMART2 risk model for recurrent cardiovascular (CV) risk in the same population. External validation was performed in 240 741 patients with ASCVD from six data sources: the Clinical Practice Research Datalink, the HUNT3 study, the SWEDEHEART Registry, the ASCVD-Particles cohort, the Estonian Biobank and the international REACH Registry. RESULTS: During a median follow-up of 11.2 years (interquartile range 6.1-16.4 years), 1031 incident HF events (13%) occurred in the UCC-SMART cohort. In the external validation data sources, a total of 24 885 incident HF events (10%) occurred. The pooled C-statistic was .696 (95% confidence interval .674-.717), with consistent performance in subgroups by sex and type of ASCVD. Predicted risks matched observed incidence in external validation. CONCLUSIONS: The SMART2-HF model enables the prediction of incident HF in patients with ASCVD. Aligned with the guideline-recommended SMART2 model for recurrent CV risk, SMART2-HF can be used as a complementary tool in this population.

Humans↗

PubMed1995

Is it clinically possible to distinguish nonhemorrhagic infarct from hemorrhagic stroke?

BACKGROUND AND PURPOSE: Diagnosis of the nonhemorrhagic ischemic type of stroke by analysis of patients' clinical features is considered unreliable because no clinical feature is specific. The diagnosis is so difficult to establish that we cannot hope to use the same method to make a reliable diagnosis in all stroke cases. In this study, we propose a simple scoring system with a positive predictive value of close to 100% to distinguish nonhemorrhagic infarct from hemorrhagic stroke. This scoring is available for all physicians in bedside diagnosis even if this score can be applied to a subgroup of patients. METHODS: Twenty-six clinical variables that might potentially distinguish cerebral hemorrhage from infarction were recorded in patients consecutively admitted to our stroke unit for stroke lasting more than 24 hours with at least unilateral motor weakness affecting face and/or arm and/or leg (internal validity study). Patients previously receiving anticoagulant therapy were excluded. We used CT scan as the gold standard. We used multivariate logistic regression to establish a clinical score from which we derived the classification rule. This rule was validated with data from the next 200 consecutive patients hospitalized in the stroke unit (external validity study). RESULTS: Three hundred sixty-eight patients were enrolled in the internal study. The obtained score was (2 x alcohol consumption) + (1.5 x plantar response) + (3 x headache) + (3 x history of hypertension)--(5 x history of transient neurological deficit)--(2 x peripheral arterial disease)--(1.5 x history of hyperlipidemia)--(2.5 x atrial fibrillation on admission). All patients with a score less than 1 (n = 123) had a nonhemorrhagic infarct (ie, 40% of the 305 patients with a nonhemorrhagic infarct). No threshold was found to diagnose cerebral hemorrhage with a sufficiently high positive predictive value. Among the 200 patients enrolled in the external validity study, 72 patients with a score below 1 had a nonhemorrhagic infarct (ie, 43% of patients with a nonhemorrhagic infarct). CONCLUSIONS: Diagnosis of nonhemorrhagic infarct can be made in 36% (95% confidence interval [CI], 29 to 43) of patients with a high level of accuracy (100% in the external validity study, which gives a 95% CI of 93 to 100). Thus, 43% (95% CI, 36 to 50) of patients with a nonhemorrhagic infarct could receive a bedside diagnosis. The score is simple and can be calculated from information available to all physicians.

Adult↗

PubMed1998

Validation of a melanoma prognostic model.

BACKGROUND: A "clinically accessible," 4-variable (patient age, patient sex, tumor location, and tumour thickness) prognostic model has been published previously. This model evaluated variables that were commonly available to the clinician. Because models are heuristic, validity of a prognostic model should be evaluated in a population different from the original population. OBJECTIVE: To evaluate the external validity of this 4-variable melanoma prognostic model. DESIGN: To estimate the external validity of this model, we used a population-based cohort of individuals with melanoma. We also evaluated a 1-variable model (tumor thickness). Estimates of the external validity of these logistic regression models were made using the c statistic and the Brier score. SETTINGS AND PATIENTS: A total of 1261 patients with melanoma evaluated in a multispecialty, university-based practice and 650 patients with melanoma from throughout Connecticut. MAIN OUTCOME MEASURE: Death from melanoma within 5 years of diagnosis. RESULTS: The c statistics for the 4-variable model were 0.86 (95% confidence interval [CI], 0.83-0.89) for the university-based practice data set and 0.81 (95% CI, 0.75-0.86) for the Connecticut data set. For thickness alone, the c statistics were 0.83 (95% CI, 0.80-0.86) and 0.79 (95% CI, 0.74-0.85), respectively. Brier scores for the 4-variable model were 0.09 (95% CI, 0.08-0.10) and 0.08 (95% CI, 0.06-0.09) and for the 1-variable model were 0.09 (95% CI, 0.08-0.10) and 0.08 (95% CI, 0.07-0.10), respectively. No significant differences exist between the data sets for the 4- and 1-variable models. CONCLUSIONS: The 4- and 1-variable models are generalizable. The simpler 1-variable model--tumor thickness--can be used with a relatively small loss in accuracy.

Female↗