Simultaneous multiple comparison procedures in psychiatric research.

Abstract

The multiple comparison problem arises in any research design in which the scores of more than two groups are compared on a single dependent variable. The analysis of variance provides a way of testing the null hypothesis that all population means are equal with a type 1 error rate of alpha. When the joint null hypothesis is accepted, the outcome of such a test is unambiguous: all population means are equal. When the null hypothesis is rejected, ANOVA is insufficient to identify the pattern of departure from equality. The Scheffe procedure can be used to test any number and type of contrast that is suggested by an inspection of the data. It ensures that the chance of incorrectly rejecting one or more hypotheses in the set so tested cannot exceed alpha. Contrasts between the population means which are specified independently of the data can be tested using Bonferroni-adjusted t tests to control the experimentwise error rate. Factorial ANOVA designs can also be analysed by testing linear contrasts. If a researcher has definite expectations about the pattern of mean differences, he can test a set of planned contrasts with a familywise error rate (if contrasts are written within main effects and interaction effects) or an EER. If he is uncertain about which hypotheses to test, post hoc procedures which are modifications of the Scheffe technique can be used.