PubMed Health⌕ Search
Home/Archive/PubMed

PubMed · 10558395

Using cluster random assignment to measure program impacts. Statistical implications for the evaluation of education programs.

H S Bloom·J M Bos·S W Lee
View original source ↗Export metadata ↓

Abstract

This article explores the possibility of randomly assigning groups (or clusters) of individuals to a program or a control group to estimate the impacts of programs designed to affect whole groups. This cluster assignment approach maintains the primary strength of random assignment--the provision of unbiased impact estimates--but has less statistical power than random assignment of individuals, which usually is not possible for programs focused on whole groups. To explore the statistical implications of cluster assignment, the authors (a) outline the issues involved, (b) present an analytic framework for studying these issues, and (c) apply this framework to assess the potential for using the approach to evaluate education programs targeted on whole schools. The findings suggest that cluster assignment of schools holds some promise for estimating the impacts of education programs when it is possible to control for the average performance of past student cohorts or the past performance of individual students.

Explore related subjects

BiasChildCluster AnalysisData Interpretation, StatisticalEducationEducational MeasurementEffect Modifier, EpidemiologicHumansLongitudinal StudiesModels, StatisticalNew YorkProgram EvaluationRandomized Controlled Trials as TopicReproducibility of ResultsResearch DesignJournal Article

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

H S Bloom, J M Bos, S W Lee. 1999. Using cluster random assignment to measure program impacts. Statistical implications for the evaluation of education programs.. https://doi.org/10.1177/0193841x9902300405

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2026

Assessment of blinding in pharmacotherapy and noninvasive neuromodulation randomized controlled trials for neuropathic pain in adults.

In randomized controlled trials (RCTs), study participants and research personnel are often blinded to minimize biases related to knowing treatment allocation. To determine if blinding was effective, participants may be asked which treatment they believe they received ("treatment guess"). This descriptive review characterized blinding assessment (BA) reporting in pharmacotherapy and neuromodulation neuropathic pain RCTs. Of 288 papers, 36 (12.5%) reported a BA. One paper reported the results of 2 studies, so in total 37 studies with a BA were assessed. Of these, 19 were crossover, 17 parallel, and 1 partial crossover in design. All 37 studies assessed participant blinding, and 10 also assessed investigator blinding. Approximately 27% included an "unsure" answer option for treatment guess, and 38% asked the reason for the guess. There were no clear patterns in BA reporting across time nor based on treatment type. Seventeen trials provided sufficient data to calculate Bang Blinding Index (BI) to determine blinding success. Participants remained blinded (BI = 0 &#xb1; 0.2) in 10/17 placebo and 10/17 treatment arms, 6 placebo and 5 treatment arms had a BI > 0.2 suggesting possible unblinding, whereas 1 placebo and 2 treatment arms had a BI < -0.2 suggesting misinformed guessing. Overall, we found that BAs are done in a minority of published neuropathic pain trials and with variable methodology. Given the importance of minimizing risk of bias because of treatment unblinding, future studies should consider including BAs, and further consensus building is necessary to determine if and how BAs should be conducted and interpreted in analgesic clinical trials.

Bias↗
PubMed2003

Sensitivity analysis for the estimation of rates of change with non-ignorable drop-out: an application to a randomized clinical trial of the vitamin D3.

The vitamin D(3) trial was a repeated measures randomized clinical trial for secondary hyperparathyroidism in haemodialysis patients where the efficacy of the vitamin D(3) infusions for suppressing the secretion of parathyroid hormone (PTH) was compared among four dose groups over 12 weeks. In this trial, patients terminated the study before the scheduled end of the study due to their elevated serum calcium (Ca) level, that is, the administration of the vitamin D(3) was expected to cause hypercalcaemia as an adverse event. In this setting of monotone missingness, there is a potential for bias in estimation of mean rates of decline in PTH for each treatment group using the standard methods such as the generalized estimating equations (GEE) which ignore the observed past Ca histories. We estimated the treatment-group-specific mean rates of decline in PTH by the inverse probability of censoring weighted (IPCW) methods which account for the observed past histories of time-dependent factors that are both a predictor of drop-out and are correlated with the outcomes. The IPCW estimator can be viewed as an extension of the GEE estimator that allows for the data to be MAR but not MCAR. With missing data, it is rarely appropriate to analyse the data solely under the assumption that the missing data process is ignorable, because the assumption of ignorable missingness cannot be guaranteed to hold and is untestable from the observed data. We proposed a sensitivity analysis that examines how inference about the IPCW estimates of the treatment-group-specific mean rates of decline in PTH changes as we vary the non-ignorable selection bias parameter over a range of plausible values.

Bias↗