PubMed Health⌕ Search

PubMed · 3206001

Identifying important results from multiple statistical tests.

Abstract

When many statistical tests are performed simultaneously, the overall chance of a type I error (incorrect rejection of a true null hypothesis) can substantially exceed the nominal error rate used in each individual test. Numerous techniques exist to adjust results of individual tests to control this problem. In general, these techniques apply a more stringent criterion of statistical significance (a smaller P-value) to each individual test than normally needed to maintain the experimentwise type I error. With an analysis that seeks to identify results for further research, however, such a conservative technique may not be appropriate. We present a new approach that uses a mixture of several distributions to model the set of P-values or of test statistics. One component models the results consistent with a failure to reject the null hypothesis, while the other distribution(s) in the mixture represent results inconsistent with the null hypothesis. These latter results may not achieve statistical significance based on a conventional P-value. We illustrate the use of the method on national mortality data and on several data sets analysed previously.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

R A Parker, R B Rothenberg. 1988. Identifying important results from multiple statistical tests.. https://doi.org/10.1002/sim.4780071005

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

Overall and cause-specific mortality in ulcerative colitis: meta-analysis of population-based inception cohort studies.

OBJECTIVES: It remains debated whether patients with ulcerative colitis (UC) are at greater risk of dying and whether a possible alteration in mortality can be attributed to specific causes of death. We aimed to clarify this issue by conducting a meta-analysis of population-based inception cohort studies on overall and cause-specific mortality in patients with UC. METHODS: The MEDLINE search engine and abstracts from international conferences were searched for relevant literature by use of explicit search criteria. STATA meta-analysis software was used to calculate pooled risk estimates (SMR, standardized mortality ratio, observed/expected deaths) of overall mortality and specific causes of death and to conduct metaregression analyses of the influence of specific variables on SMR. RESULTS: Ten papers fulfilled the inclusion criteria, reporting SMRs varying from 0.7 to 1.4. The overall pooled estimate was 1.1 (95% confidence interval [CI] 0.9-1.2, P= 0.42). However, greater risk of dying was observed during the first years of follow-up, in patients with extensive colitis, and in patients from Scandinavia. Metaregression analysis showed an increase in SMR by increasing cohort size. UC-related mortality accounted for 17% of all deaths. Mortality from gastrointestinal diseases, nonalcoholic liver diseases, pulmonary embolisms, and respiratory diseases was increased whereas mortality from pulmonary cancer was reduced. CONCLUSIONS: The overall risk of dying in patients with UC did not differ from that of the background population, although subgroups of patients were at greater risk of dying. The cause-of-death distribution seemed to differ from that of the background population.

Cause of Death↗