PubMed Health⌕ Search

SEARCH · PubMed Health

Results for “multiple testing”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 recordsLinked to original sources
PubMed2001

Adjusting for multiple testing--when and how?

Multiplicity of data, hypotheses, and analyses is a common problem in biomedical and epidemiological research. Multiple testing theory provides a framework for defining and controlling appropriate error rates in order to protect against wrong conclusions. However, the corresponding multiple test procedures are underutilized in biomedical and epidemiological research. In this article, the existing multiple test procedures are summarized for the most important multiplicity situations. It is emphasized that adjustments for multiple testing are required in confirmatory studies whenever results from multiple tests have to be combined in one final conclusion and decision. In case of multiple significance tests a note on the error rate that will be controlled for is desirable.

Bias↗
PubMed1989

A simplified approach for evaluating multiple test outcomes and multiple disease states in relation to the exercise thallium-201 stress test in suspected coronary artery disease.

This study describes a simplified approach for the interpretation of electrocardiographic and thallium-201 imaging data derived from the same patient during exercise. The 383 patients in this study had also undergone selective coronary arteriography within 3 months of the exercise test. This matrix approach allows for multiple test outcomes (both tests positive, both negative, 1 test positive and 1 negative) and multiple disease states (no coronary artery disease vs 1-vessel vs multivessel coronary artery disease). Because this approach analyzes the results of 2 test outcomes simultaneously rather than serially, it also negates the lack of test independence, if such an effect is present. It is also demonstrated that ST-segment depression on the electrocardiogram and defects on initial thallium-201 images provide conditionally independent information regarding the presence of coronary artery disease in patients without prior myocardial infarction. In contrast, ST-segment depression on the electrocardiogram and redistribution on the delayed thallium-201 images may not provide totally independent information regarding the presence of exercise-induced ischemia in patients with or without myocardial infarction.

Aged↗
PubMed2006

An alternative foundation for the planning and evaluation of linkage analysis. II. Implications for multiple test adjustments.

The 'multiple testing problem' currently bedevils the field of genetic epidemiology. Briefly stated, this problem arises with the performance of more than one statistical test and results in an increased probability of committing at least one Type I error. The accepted/conventional way of dealing with this problem is based on the classical Neyman-Pearson statistical paradigm and involves adjusting one's error probabilities. This adjustment is, however, problematic because in the process of doing that, one is also adjusting one's measure of evidence. Investigators have actually become wary of looking at their data, for fear of having to adjust the strength of the evidence they observed at a given locus on the genome every time they conduct an additional test. In a companion paper in this issue (Strug & Hodge I), we presented an alternative statistical paradigm, the 'evidential paradigm', to be used when planning and evaluating linkage studies. The evidential paradigm uses the lod score as the measure of evidence (as opposed to a p value), and provides new, alternatively defined error probabilities (alternative to Type I and Type II error rates). We showed how this paradigm separates or decouples the two concepts of error probabilities and strength of the evidence. In the current paper we apply the evidential paradigm to the multiple testing problem - specifically, multiple testing in the context of linkage analysis. We advocate using the lod score as the sole measure of the strength of evidence; we then derive the corresponding probabilities of being misled by the data under different multiple testing scenarios. We distinguish two situations: performing multiple tests of a single hypothesis, vs. performing a single test of multiple hypotheses. For the first situation the probability of being misled remains small regardless of the number of times one tests the single hypothesis, as we show. For the second situation, we provide a rigorous argument outlining how replication samples themselves (analyzed in conjunction with the original sample) constitute appropriate adjustments for conducting multiple hypothesis tests on a data set.

Chromosome Mapping↗
PubMed2006

A two-stage design for multiple testing in large-scale association studies.

Modern association studies often involve a large number of markers and hence may encounter the problem of testing multiple hypotheses. Traditional procedures are usually over-conservative and with low power to detect mild genetic effects. From the design perspective, we propose a two-stage selection procedure to address this concern. Our main principle is to reduce the total number of tests by removing clearly unassociated markers in the first-stage test. Next, conditional on the findings of the first stage, which uses a less stringent nominal level, a more conservative test is conducted in the second stage using the augmented data and the data from the first stage. Previous studies have suggested using independent samples to avoid inflated errors. However, we found that, after accounting for the dependence between these two samples, the true discovery rate increases substantially. In addition, the cost of genotyping can be greatly reduced via this approach. Results from a study of hypertriglyceridemia and simulations suggest the two-stage method has a higher overall true positive rate (TPR) with a controlled overall false positive rate (FPR) when compared with single-stage approaches. We also report the analytical form of its overall FPR, which may be useful in guiding study design to achieve a high TPR while retaining the desired FPR.

Cost-Benefit Analysis↗
PubMed1996

A permutational step-up method of testing multiple outcomes.

This paper describes a permutational step-up multiple comparison procedure to adjust the p values from k related hypotheses. The method is applicable when two groups of subjects are being compared on each of k outcomes. It is related to the step-down method of Westfall and Young (1993, Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment, Volume 1) and Troendle (1995, Journal of the American Statistical Association 90, 370-378) and is an alternative to the analytic step-up method of Dunnett and Tamhane (1992, Journal of the American Statistical Association 87, 162-170), which requires a specific distribution and correlation structure. By conditioning on the data observed, this permutational method avoids any specific distribution or correlation assumption. It is shown very generally that the method asymptotically controls the familywise probability of a type I error.

Biometry↗
PubMed1991

A note on multiple testing procedures in linkage analysis.

Controversy over the impact of multiple testing procedures in linkage analysis is reexamined in this report. Despite some recent claims to the contrary, it is shown that testing multiple markers decreases the posterior false-positive rate among significant tests, rather than increasing it; this is true whether the trait of interest is simply monogenic or complex, or even if the genetic model is misspecified. However, if the true mode of inheritance is complex, or if the genetic model is misspecified, the power to obtain a significant result when linkage is present may be reduced, while the significance level is not, leading to an inflation of the posterior false-positive rate. Furthermore, the posterior false-positive rate increases with decreasing sample size and may be unacceptably high for very small samples. By contrast, testing multiple genetic models, by varying either mode-of-inheritance parameters or diagnostic categories, does lead to an inflation of the posterior false-positive rate. A conservative correction for this case is to subtract log10t from the obtained maximum lod score, where t different genetic and/or diagnostic models have been tested.

Genetic Linkage↗
PubMed2000

A test for spatial disease clustering adjusted for multiple testing.

Tango (1995) proposed a test statistic C for detecting spatial disease clusters. However, like most other methods, it requires a value of the scale parameter which adjusts for the size of cluster to be detected in advance of its use. When an appropriate size of cluster cannot be predicted and many clustered areas are expected, we usually repeat the procedure by changing the size parameter, but this practice clearly faces a multiple testing problem. In this paper, to ameliorate this problem, we propose an extended test statistic which searches the minimum of the profile P-value of C for the parameter on cluster size which varies continuously from a small value near zero upwards until it reaches to about half the size of the whole study area. Monte Carlo simulation study shows that the power of the proposed method is shown to be reasonably high compared with the best power attained by C unadjusted for multiple testing in all the cluster models considered. The proposed procedure is illustrated with some disease maps simulated in the Tokyo Metropolitan Area.

Humans↗
PubMed1996

Single and multiple test repetition priming in implicit memory.

Using a picture naming task, we compared the magnitude of repetition priming after one prior study episode (single test priming) versus multiple prior study presentations (multiple test priming). Pictures were repeated either one, two, or three times, and the interval between tests was either several minutes (blocked test) or one week (spaced test). Priming increased with additional prior presentations (beyond one) in the multiple test format. In addition, single test priming decreased within one hour after initial exposure, with little change from one hour to two weeks. Priming was unaffected by a simultaneous recognition task, suggesting that picture naming is an implicit task relatively immune from explicit memory contamination.

Humans↗
PubMed2005

[The problem of multiple testing and solutions for genome-wide studies].

The problem of multiple testing and its solutions for genome-wide studies. Even if there is no real change, the traditional p = 0.05 can cause 5% of the investigated tests being reported significant. Multiple testing corrections have been developed to solve this problem. Here the authors describe the one-step (Bonferroni), multi-step (step-down and step-up) and graphical methods. However, sometimes a correction for multiple testing creates more problems, than it solves: the universal null hypothesis is of little interest, the exact number of investigations to be adjusted for can not determined and the probability of type II error increases. For these reasons the authors suggest not to perform multiple testing corrections routinely. The calculation of the false discovery rate is a new method for genome-wide studies. Here the p value is substituted by the q value, which also shows the level of significance. The q value belonging to a measurement is the proportion of false positive measurements when we accept it as significant. The authors propose using the q value instead of the p value in genome-wide studies.

Confidence Intervals↗
PubMed2006

A method to increase the power of multiple testing procedures through sample splitting.

Consider the standard multiple testing problem where many hypotheses are to be tested, each hypothesis is associated with a test statistic, and large test statistics provide evidence against the null hypotheses. One proposal to provide probabilistic control of Type-I errors is the use of procedures ensuring that the expected number of false positives does not exceed a user-supplied threshold. Among such multiple testing procedures, we derive the most powerful method, meaning the test statistic cutoffs that maximize the expected number of true positives. Unfortunately, these optimal cutoffs depend on the true unknown data generating distribution, so could never be used in a practical setting. We instead consider splitting the sample so that the optimal cutoffs are estimated from a portion of the data, and then testing on the remaining data using these estimated cutoffs. When the null distributions for all test statistics are the same, the obvious way to control the expected number of false positives would be to use a common cutoff for all tests. In this work, we consider the common cutoff method as a benchmark multiple testing procedure. We show that in certain circumstances the use of estimated optimal cutoffs via sample splitting can dramatically outperform this benchmark method, resulting in increased true discoveries, while retaining Type-I error control. This paper is an updated version of the work presented in Rubin et al. (2005), later expanded upon by Wasserman and Roeder (2006).

Algorithms↗
PubMed1979

A multiple testing procedure for clinical trials.

A multiple testing procedure is proposed for comparing two treatments when response to treatment is both dichotomous (i.e., success or failure) and immediate. The proposed test statistic for each test is the usual (Pearson) chi-square statistic based on all data collected to that point. The maximum number (N) of tests and the number (m1 + m2) of observations collected between successive tests is fixed in advance. The overall size of the procedure is shown to be controlled with virtually the same accuracy as the single sample chi-square test based on N(m1 + m2) observations. The power is also found to be virtually the same. However, by affording the opportunity to terminate early when one treatment performs markedly better than the other, the multiple testing procedure may eliminate the ethical dilemmas that often accompany clinical trials.

Clinical Trials as Topic↗
PubMed2006

Powers of multiple-testing procedures for identification of genes significantly differentially expressed in microarray experiments.

Because of the high operation costs involved in microarray experiments, the determination of the number of replicates required to detect a gene significantly differentially expressed in a given multiple-testing procedure is of considerable significance. Calculation of power/replicate numbers required in multiple-testing procedures provides design guidance for microarray experiments. Based on this model and by choice of a multiple-testing procedure, expression noises based on permutation resampling can be considerably minimized. The method for mixture distribution model is suitable to various microarray data types obtained from single noise sources, or from multiple noise sources. By using the biological replicate number required in microarray experiments for a given power or by determining the power required to detect a gene significantly differentially expressed, given the sample size, or the best multiple-testing method can be chosen. As an example, a single-distribution model of t-statistic was fitted to an observed microarray dataset of 3 000 genes responsive to stroke in rat, and then used to calculate powers of four popular multiple-testing procedures to detect a gene of an expression change D. The results show that the B-procedure had the lowest power to detect a gene of small change among the multiple-testing procedures, whereas the BH-procedure had the highest power. However, all multiple-testing procedures had the same power to identify a gene having the largest change. Similar to a single test, the power of the BH-procedure to detect a small change does not vary as the number of genes increases, but powers of the other three multiple-testing procedures decline as the number of genes increases.

Gene Expression Profiling↗
PubMed2005

Rapid simulation of P values for product methods and multiple-testing adjustment in association studies.

A major aim of association studies is the identification of polymorphisms (usually SNPs) associated with a trait. Tests of association may be based on individual SNPs or on sets of neighboring SNPs, by use of (for example) a product P value method or Hotelling's T test. Linkage disequilibrium, the nonindependence of SNPs in physical proximity, causes problems for all these tests. First, multiple-testing correction for individual-SNP tests or for multilocus tests either leads to conservative P values (if Bonferroni correction is used) or is computationally expensive (if permutation is used). Second, calculation of product P values usually requires permutation. Here, we present the direct simulation approach (DSA), a method that accurately approximates P values obtained by permutation but is much faster. It may be used whenever tests are based on score statistics--for example, with Armitage's trend test or its multivariate analogue. The DSA can be used with binary, continuous, or count traits and allows adjustment for covariates. We demonstrate the accuracy of the DSA on real and simulated data and illustrate how it might be used in the analysis of a whole-genome association study.

Antidepressive Agents↗