Ensuring the comparability of comparison groups: is randomization enough?

Abstract

BACKGROUND: It is widely believed that baseline imbalances in randomized trials must necessarily be random. In fact, there is a type of selection bias that can cause substantial, systematic and reproducible baseline imbalances of prognostic covariates even in properly randomized trials. It is possible, given complete data, to quantify both the susceptibility of a given trial to this type of selection bias and the extent to which selection bias appears to have caused either observable or unobservable baseline imbalances. Yet, in articles reporting on randomized trials, it is uncommon to find either these assessments or the information that would enable a reader to conduct them. Nevertheless, there have been a few published reports that contain descriptions of either this type of selection bias or indicators that it may have occurred. OBJECTIVE: To document that the same type of selection bias has been described in numerous randomized trials and therefore that it represents a problem deserving of greater attention. STUDY SELECTION: Computerized searches were not useful in locating trials with one or more elements that contribute to or are indicative of selection bias in randomized trials. We limit our treatment to trials that were previously questioned for susceptibility to selection bias or for large baseline imbalances. RESULTS: We found 14 randomized trials that appear to be suspicious for selection bias. This may represent only the tip of the iceberg, because the status of other trials is inconclusive. CONCLUSIONS: Authors of clinical trial reports should be required to disclose sufficient details to allow for an assessment of both allocation concealment and selection bias. The extent to which a randomized study was susceptible to selection bias should be considered in determining the relative contribution it makes to any subsequent meta-analysis, policy or decision.