A method to quantify rater bias in antidepressant trials.
Some studies indicate that the blind in clinical trials of the efficacy of antidepressant drugs is less than perfect. It has been suggested that, as a consequence of this incomplete blind, biased raters inflate efficacy and that, in fact, these drugs are relatively ineffective. However, in the literature, we could find no prior attempt to quantify rater bias and, thus, measure its contribution to claims of antidepressant efficacy. We used the distribution of SCL-90 (Symptom Check List) depression scale scores to derive a patient-based effect size, and contrasted this with the clinician-based effect size. We propose the difference between these two effect sizes (patient self-rating and clinician-derived) to be an indirect measure of bias. If patients had a prodrug bias, this method would be invalid. However the response rate from studies with active placebo suggest a patient prodrug bias is unlikely. The effect sizes derived from patient self-ratings are smaller than those derived from clinician ratings. This allows for the possibility that some clinician ratings were biased. However, quantifying the effect of bias suggests that it was insufficient to invalidate the original study conclusions based on clinician ratings, because the proportion of responders, based on patient self-ratings, differed significantly between the two drugs and placebo. Their 95% confidence intervals (CI) did not overlap. This analysis allows that some clinician ratings may be biased. However, the extent of bias appears insufficient to alter conclusions based on clinician ratings regarding efficacy of antidepressant drugs in this trial. Application of our approach in other trials is necessary to establish generalizability.