Uniform power method for sample size calculation in historical control studies with binary response.

Abstract

Makuch and Simon gave a sample size calculation formula for historical control (HC) studies that assumed that the observed response rate in the control group is the true response rate. We dropped this assumption and computed the expected power and expected sample size to evaluate the performance of the procedure under the omniscient model. When there is uncertainty in the HC response rate but this uncertainty is not considered, Makuch and Simon's method produces a sample size that gives a considerably lower power than that specified. Even the larger sample size obtained from the randomized design formula and applied to the HC setting does not guarantee the advertised power in the HC setting. We developed a new uniform power method to search for the sample size required for the experimental group to yield an exact power without relying on the estimated HC response rate being perfectly correct. The new method produces the correct uniform predictive power for all permissible response rates. The resulting sample size is closer to the sample size needed for the randomized design than Makuch and Simon's method, especially when there is a small difference in response rates or a limited sample size in the HC group. HC design may be a viable option in clinical trials when the patient selection bias and the outcome evaluation bias can be minimized. However, the common perception of the extra sample size savings is largely unjustified without the strong assumption that the observed HC response rate is equal to the true control response rate. Generally speaking, results from HC studies need to be confirmed by studies with concurrent controls and cannot be used for making definitive decisions.