P-values after repeated significance testing: a simple approximation method.

Abstract

The P-value after a repeated significance test is a useful measure of the strength of evidence against the null hypothesis. Its computation, however, requires a computer-intensive numerical integration method. The P-value is not conceptually straightforward, because it depends on how the sample space is ordered, which can be arbitrary. We look at two orderings of the sample space, one proposed by Tsiatis et al. and the other by Rosner and Tsiatis, and Chang. Although studies have shown that the latter ordering gives more reasonable confidence intervals than the former, the former gives a conservative and therefore more reasonable P-value. Both, however, should yield an identical P-value in most applications. In this paper we present a simple method of approximating P-values. We provide tables to implement the method for two to ten stages with alpha = 0.1, 0.05 and 0.01 for the Pocock and O'Brien-Fleming procedures. The proposed method can be applied to both orderings.