Meta-analysis of binary data: which within study variance estimate to use?

Abstract

We applied a mixed effects model to investigate between- and within-study variation in improvement rates of 180 schizophrenia outcome studies. The between-study variation was explained by the fixed study characteristics and an additional random study effect. Both rate difference and logit models were used. For a binary proportion outcome p(i) with sample size n(i) in the ith study, (circumflexp(i)(1-circumflexp(i))n)(-1) is the usual estimate of the within-study variance sigma(i)(2) in the logit model, where circumflexpi) is the sample mean of the binary outcome for subjects in study i. This estimate can be highly correlated with logit(circumflexp(i)). We used (macronp(i)(1-macronp)n(i))(-1) as an alternative estimate of sigma(i)(2), where macronp is the weighted mean of circumflexp(i)'s. We estimated regression coefficients (beta) of the fixed effects and the variance (tau(2)) of the random study effect using a quasi-likelihood estimating equations approach. Using the schizophrenia meta-analysis data, we demonstrated how the choice of the estimate of sigma(2)(i) affects the resulting estimates of beta and tau(2). We also conducted a simulation study to evaluate the performance of the two estimates of sigma(2)(i) in different conditions, where the conditions vary by number of studies and study size. Using the schizophrenia meta-analysis data, the estimates of beta and tau(2) were quite different when different estimates of sigma(2)(i) were used in the logit model. The simulation study showed that the estimates of beta and tau(2) were less biased, and the 95 per cent CI coverage was closer to 95 per cent when the estimate of sigma(2)(i) was (macronp(1-macronp)n(i))(-1) rather than (circumflexp(i)(1-circumflexp)n(i))(-1). Finally, we showed that a simple regression analysis is not appropriate unless tau(2) is much larger than sigma(2)(i), or a robust variance is used.