[Examples of pitfalls in statistical analysis--3: Why do we need to use multiple comparison procedures?].

Abstract

When comparing the means of more than three groups, we first have to carry out an analysis of variance (ANOVA) to confirm whether all of the means of each group are equal, and we then have to evaluate all possible pairwise comparisons with multiple comparison procedures. If we use only the unpaired t-test for all pairwise comparisons without ANOVA and multiple comparison procedures, we are more likely not only to erroneously conclude that two means in a pair are significantly different when actually they are not, but also to create a contradiction between pairs of means with large differences which are not considered significant, and pairs of means with small differences which are considered significant. In this article, I discuss two problems in comparing the means of multiple groups by presenting an example in which I apply unpaired t-test on data that should be analyzed by ANOVA and multiple comparison procedures.