PubMed Health⌕ Search

PubMed · 9888002

Multiple comparison procedures updated.

Abstract

1. A common statistical flaw in articles submitted to or published in biomedical research journals is to test multiple null hypotheses that originate from the results of a single experiment without correcting for the inflated risk of type 1 error (false positive statistical inference) that results from this. Multiple comparison procedures (MCP) are designed to minimize this risk. The present review focuses on pairwise contrasts, the most common sort of multiple comparisons made by biomedical investigators. 2. In an earlier review a variety of MCP were described and evaluated. It was concluded that an effective MCP should control the risk of family-wise type 1 error, so as to ensure that not more than one hypothesis within a single family is falsely rejected. One-step procedures based on the Bonferroni or Sidák inequalities do this. For continuous data and under normal distribution theory, so does the Tukey-Kramer procedure for all possible pairwise contrasts of means and the Dunnett procedure for all possible pairwise contrasts of means with a control mean. 3. There is now a new class of MCP, based on the Bonferroni or Sidák inequalities but performed in a step-wise fashion. The members of this class have certain desirable properties. They: (i) control the family-wise type 1 error rate as effectively as the one-step procedures; (ii) are more powerful than the one-step Bonferroni or Sidák procedures, especially when hypotheses are logically related; and (iii) can be applied not only to continuous data but also to ordinal or categorical data. 4. Of the new step-wise MCP, Holm's step-down procedures are commended for their combination of accuracy, power and versatility. They also have the virtue of simplicity. Given the raw P values that result from conventional tests of significance, the adjustments for multiple comparisons can be made by hand or hand-held calculator. 5. Despite the corrective abilities of the new step-wise MCP, investigators should try to design their experiments and analyses to test a single, global hypothesis rather than multiple ones.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

J Ludbrook. 1998. Multiple comparison procedures updated.. https://doi.org/10.1111/j.1440-1681.1998.tb02179.x

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

Molecular heterochrony and the evolution of sociality in bumblebees (Bombus terrestris).

Sibling care is a hallmark of social insects, but its evolution remains challenging to explain at the molecular level. The hypothesis that sibling care evolved from ancestral maternal care in primitively eusocial insects has been elaborated to involve heterochronic changes in gene expression. This elaboration leads to the prediction that workers in these species will show patterns of gene expression more similar to foundress queens, who express maternal care behaviour, than to established queens engaged solely in reproductive behaviour. We tested this idea in bumblebees (Bombus terrestris) using a microarray platform with approximately 4500 genes. Unlike the wasp Polistes metricus, in which support for the above prediction has been obtained, we found that patterns of brain gene expression in foundress and queen bumblebees were more similar to each other than to workers. Comparisons of differentially expressed genes derived from this study and gene lists from microarray studies in Polistes and the honeybee Apis mellifera yielded a shared set of genes involved in the regulation of related social behaviours across independent eusocial lineages. Together, these results suggest that multiple independent evolutions of eusociality in the insects might have involved different evolutionary routes, but nevertheless involved some similarities at the molecular level.

Analysis of Variance↗

Confidence intervals for the standardized effect arising in the comparison of two normal populations.

Confidence intervals for a standardized effect are derived after stabilizing the variance of the Welch t-statistic. Simulation studies demonstrate the viability of the resulting intervals for a wide range of parameter values and sample sizes as small as five. The methodology is extended to the combination of results from several studies, so as to obtain a confidence interval for a representative standardized effect for all the studies. The methods are illustrated on a recent meta-analytic study of systolic blood pressure reduction during a weight reducing regime, as well as the classical Mumford data on psychological intervention and hospital length of stay.

Analysis of Variance↗