SEARCH · PubMed Health

Results for “multiple testing”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMed1981

Multiple tests in the diagnosis of multiple sclerosis.

Visual evoked potentials (VEPs), somatosensory evoked potentials (SEPs), E-UFA test and CSF isoelectrofocusing (IEF) were applied to 42 multiple sclerosis patients. VEPs were abnormal in 29 cases. SEPs in 31 cases, E-UFA in 30 cases, CSF IEF in 38 cases. At least two tests were abnormal in 41 out of 42 patients; 16 cases revealed pathological responses in three tests (38,1%), 15 in four of them (35,7%). The authors discuss the value of using several tests in the diagnosis of multiple sclerosis and emphasize their importance for the purpose of diagnostic classification.

Adolescent↗

PubMed2005

Multiple testing in the context of haplotype analysis revisited: application to case-control data.

We have lately presented a testing procedure for family data which accounts for the multiple testing problem that is induced by the enormous number of different marker combinations that can be analyzed in a set of tightly linked markers. Most methods of haplotype based association analysis already require simulations to obtain an uncorrected P value for a specific marker combination. As shown before, it is nevertheless not necessary to carry out nested simulations to obtain a global P value that properly corrects for the multiple testing of different marker combinations without neglecting the dependency of the tests. We have now implemented this approach for case-control data in our program FAMHAP, as this data structure currently plays a dominant role in the field. We consider different ways to deal with phase ambiguities and two different statistical tests for the underlying single marker combinations to obtain uncorrected P values. One test statistic is chi-square based, the other is a haplotype trend regression. The performance of these different tests in the multiple testing situation is investigated in a large simulation study. We obtain a considerable gain in power with our global P values as opposed to Bonferroni corrected P values for all suggested test statistics. Good power was obtained both with the haplotype trend regression approach as well as with the simpler chi-square based test. Furthermore, we conclude that the better strategy to deal with phase ambiguities is to assign to each individual its list of weighted haplotype explanations, rather than to assign to each individual its most likely haplotype explanation. Finally, we demonstrate the usefulness of our approach by a real data example.

Algorithms↗

PubMed2006

Testing multiple statistical hypotheses resulted in spurious associations: a study of astrological signs and health.

OBJECTIVES: To illustrate how multiple hypotheses testing can produce associations with no clinical plausibility. STUDY DESIGN AND SETTING: We conducted a study of all 10,674,945 residents of Ontario aged between 18 and 100 years in 2000. Residents were randomly assigned to equally sized derivation and validation cohorts and classified according to their astrological sign. Using the derivation cohort, we searched through 223 of the most common diagnoses for hospitalization until we identified two for which subjects born under one astrological sign had a significantly higher probability of hospitalization compared to subjects born under the remaining signs combined (P<0.05). RESULTS: We tested these 24 associations in the independent validation cohort. Residents born under Leo had a higher probability of gastrointestinal hemorrhage (P=0.0447), while Sagittarians had a higher probability of humerus fracture (P=0.0123) compared to all other signs combined. After adjusting the significance level to account for multiple comparisons, none of the identified associations remained significant in either the derivation or validation cohort. CONCLUSIONS: Our analyses illustrate how the testing of multiple, non-prespecified hypotheses increases the likelihood of detecting implausible associations. Our findings have important implications for the analysis and interpretation of clinical studies.

Adolescent↗

PubMed1991

Multiple testing in clinical trials.

The basic ideas of multiple testing are outlined and the problem of how to control the probability of erroneous decisions is discussed. The main emphasis is on the concept of the multiple level of significance (controlling the experiment, or family error in the strong sense) which can be achieved by applying the principle of closed tests. Various practical situations encountered in multiple testing in clinical trials are considered: more than one end point; more than two treatments, such as comparisons with a single control, comparisons using ordered alternatives, all pairwise comparisons and contrast methods; and more than one trial. Tests based on global statistics, the union intersection principle and other criteria are discussed. The application of the multiple test concept in sequential sampling is investigated. Finally some comments are made on multiple power, multiple confidence intervals and directed decisions.

Analysis of Variance↗

PubMed1994

A rapid algorithm and a computer program for multiple test procedures using logical structures of hypotheses.

It is demonstrated how improvements of general multiple test procedures can be obtained using information about the logical structures among the hypotheses. Based on a procedure of Bergmann and Hommel (B. Bergmann and G. Hommel, Improvements of general multiple test procedures for redundant systems of hypotheses, in Multiple Hypothesenprüfung--Multiple Hypotheses Testing, Eds. P. Bauer, G. Hommel and E. Sonnemann, pp. 100-115 (Springer-Verlag, Berlin, 1988)), a computer program was written by Bernhard (G. Bernhard, Computerunterstützte Durchführung von multiplen Testprozeduren--Algorithmen und Powervergleich, Doctoral thesis (Mainz, 1992)) using this information. It is applicable for a general class of systems of hypotheses which can be expressed in a linear way. By means of a simulation study it is shown that the proposed procedure is often substantially more powerful than other usual multiple test procedures.

Algorithms↗

PubMed2006

Quantile-function based null distribution in resampling based multiple testing.

Simultaneously testing a collection of null hypotheses about a data generating distribution based on a sample of independent and identically distributed observations is a fundamental and important statistical problem involving many applications. Methods based on marginal null distributions (i.e., marginal p-values) are attractive since the marginal p-values can be based on a user supplied choice of marginal null distributions and they are computationally trivial, but they, by necessity, are known to either be conservative or to rely on assumptions about the dependence structure between the test-statistics. Re-sampling based multiple testing (Westfall and Young, 1993) involves sampling from a joint null distribution of the test-statistics, and controlling (possibly in a, for example, step-down fashion) the user supplied type-I error rate under this joint null distribution for the test-statistics. A generally asymptotically valid null distribution avoiding the need for the subset pivotality condition for the vector of test-statistics was proposed in Pollard, van der Laan (2003) for null hypotheses about general real valued parameters. This null distribution was generalized in Dudoit, vanderLaan, Pollard (2004) to general null hypotheses and test-statistics. In ongoing recent work van der Laan, Hubbard (2005), we propose a new generally asymptotically valid null distribution for the test-statistics and a corresponding bootstrap estimate, whose marginal distributions are user supplied, and can thus be set equal to the (most powerful) marginal null distributions one would use in univariate testing to obtain a p-value. Previous proposed null distributions either relied on a restrictive subset pivotality condition (Westfall and Young) or did not guarantee this latter property (Dudoit, vanderLaan, Pollard, 2004). It is argued and illustrated that the resulting new re-sampling based multiple testing methods provide more accurate control of the wished Type-I error in finite samples and are more powerful. We establish formal results and investigate the practical performance of this methodology in a simulation and data analysis.

Chi-Square Distribution↗

PubMed1994

The performance of the O'Brien-Fleming multiple testing procedure in the presence of intraclass correlation.

Assuming that all subject responses were independent, O'Brien and Fleming (1979, Biometrics 35, 549-556) proposed a simple and useful multiple testing procedure for clinical trials for comparing two treatments with dichotomous data. Differences in the methods of evaluating subject responses at each evaluation time, however, may induce an intraclass correlation among these responses used in calculating the O'Brien-Fleming multiple testing procedure. On the basis of Monte Carlo simulations, we note that even a small intraclass correlation among subject responses in the same analysis can substantially inflate the Type I error of the O'Brien-Fleming multiple testing procedure. Furthermore, this inflation generally increases as either the number of analyses or the underlying response probability increases. We also have demonstrated that if we were able to maintain a uniform medical test procedure between the two treatments for each analysis, the actual Type I error of the O'Brien-Fleming multiple testing procedure may conversely become conservative.

Analysis of Variance↗

PubMed1989

A multiple testing approach for hazard evaluation of complex mixtures in the aquatic environment: the use of diesel oil as a model.

Traditional single species toxicity tests and multiple component laboratory-scaled microcosm assays were combined to assess the toxicological hazard of diesel oil, a model complex mixture, to a model aquatic environment. The immediate impact of diesel oil dosed on a freshwater community was studied in a model pond microcosm over 14 days: a 7-day dosage and a 7-day recovery period. A multicomponent laboratory microcosm was designed to monitor the biological effects of diesel oil (1.0 mg litre(-1)) on four components: water, sediment (soil + microbiota), plants (aquatic macrophytes and algae), and animals (zooplanktonic and zoobenthic invertebrates). To determine the sensitivity of each part of the community to diesel oil contamination and how this model community recovered when the oil dissipated, limnological, toxicological, and microbiological variables were considered. Our model revealed these significant occurrences during the spill period: first, a community production and respiration perturbation, characterized in the water column by a decrease in dissolved oxygen and redox potential and a concomitant increase in alkalinity and conductivity; second, marked changes in microbiota of sediments that included bacterial heterotrophic dominance and a high heterotrophic index (0.6), increased bacterial productivity, and the marked increases in numbers of saprophytic bacteria (10 x) and bacterial oil degraders (1000 x); and third, column water acutely toxic (100% mortality) to two model taxa: Selenastrum capricornutum and Daphnia magna. Following the simulated clean-up procedure to remove the oil slick, the recovery period of this freshwater microcosm was characterized by a return to control values. This experimental design emphasized monitoring toxicological responses in aquatic microcosm; hence, we proposed the term 'toxicosm' to describe this approach to aquatic toxicological hazard evaluation. The toxicosm as a valuable toxicological tool for screening aquatic contaminants was demonstrated using diesel oil as a model complex mixture.

Journal Article↗

PubMed2005

Adjusting multiple testing in multilocus analyses using the eigenvalues of a correlation matrix.

Correlated multiple testing is widely performed in genetic research, particularly in multilocus analyses of complex diseases. Failure to control appropriately for the effect of multiple testing will either result in a flood of false-positive claims or in true hits being overlooked. Cheverud proposed the idea of adjusting correlated tests as if they were independent, according to an 'effective number' (M(eff)) of independent tests. However, our experience has indicated that Cheverud's estimate of the Meff is overly large and will lead to excessively conservative results. We propose a more accurate estimate of the M(eff), and design M(eff)-based procedures to control the experiment-wise significant level and the false discovery rate. In an evaluation, based on both real and simulated data, the M(eff)-based procedures were able to control the error rate accurately and consequently resulted in a power increase, especially in multilocus analyses. The results confirm that the M(eff) is a useful concept in the error-rate control of correlated tests. With its efficiency and accuracy, the M(eff) method provides an alternative to computationally intensive methods such as the permutation test.

Data Interpretation, Statistical↗

PubMed2004

Multiple testing. Part I. Single-step procedures for control of general type I error rates.

The present article proposes general single-step multiple testing procedures for controlling Type I error rates defined as arbitrary parameters of the distribution of the number of Type I errors, such as the generalized family-wise error rate. A key feature of our approach is the test statistics null distribution (rather than data generating null distribution) used to derive cut-offs (i.e., rejection regions) for these test statistics and the resulting adjusted p-values. For general null hypotheses, corresponding to submodels for the data generating distribution, we identify an asymptotic domination condition for a null distribution under which single-step common-quantile and common-cut-off procedures asymptotically control the Type I error rate, for arbitrary data generating distributions, without the need for conditions such as subset pivotality. Inspired by this general characterization of a null distribution, we then propose as an explicit null distribution the asymptotic distribution of the vector of null value shifted and scaled test statistics. In the special case of family-wise error rate (FWER) control, our method yields the single-step minP and maxT procedures, based on minima of unadjusted p-values and maxima of test statistics, respectively, with the important distinction in the choice of null distribution. Single-step procedures based on consistent estimators of the null distribution are shown to also provide asymptotic control of the Type I error rate. A general bootstrap algorithm is supplied to conveniently obtain consistent estimators of the null distribution. The special cases of t- and F-statistics are discussed in detail. The companion articles focus on step-down multiple testing procedures for control of the FWER (van der Laan et al., 2004b) and on augmentations of FWER-controlling methods to control error rates such as tail probabilities for the number of false positives and for the proportion of false positives among the rejected hypotheses (van der Laan et al., 2004a). The proposed bootstrap multiple testing procedures are evaluated by a simulation study and applied to genomic data in the fourth article of the series (Pollard et al., 2004).

Journal Article↗

PubMed2005

Multiple testing procedures based on weighted Kaplan-Meier statistics for right-censored survival data.

In clinical trials or drug development studies, researchers are often interested in identifying which treatments or dosages are more effective than the standard one. Recently, several multiple testing procedures based on weighted logrank tests have been proposed to compare several treatments with a control in a one-way layout where survival data are subject to random right-censorship. However, weighted logrank tests are based on ranks, and these tests might not be sensitive to the magnitude of the difference in survival times against a specific alternative. Therefore, it is desirable to develop a more robust and powerful multiple testing procedure. This paper proposes multiple testing procedures based on two-sample weighted Kaplan-Meier statistics, each comparing an individual treatment with the control, to determine which treatments are more effective than the control. The comparative results from a simulation study are presented and the implementation of these methods to the prostate cancer clinical trial and the renal carcinoma tumour study are presented.

Administration, Oral↗

PubMed2001

[Dangers of sub-group analysis and of multiple tests].

Subgroup analyses, "retrospective" statistics, and multiple tests, motivated by the search for explanatory covariables, are frequently used in the medical literature. However, these data manipulations induce modifications of the groups defined by initial randomization, an unverifiable increase of the alpha risk, a reduction of the power of the study and a risk of circular reasoning. Taking the example of an article concerning the hormonal treatment of prostate cancer, several situations are discussed, according to whether or not the study is prospective, whether or not the covariables are defined from the outset, and whether or not the results for the primary endpoint are significant.

Humans↗

PubMed2000

Circumventing multiple testing: a multilocus Monte Carlo approach to testing for association.

Advances in marker technology have made a dense marker map a reality. If each marker is considered separately, and separate tests for association with a disease gene are performed, then multiple testing becomes an issue. A common solution uses a Bonferroni correction to account for multiple tests performed. However, with dense marker maps, neighboring markers are tightly linked and may have associated alleles; thus tests at nearby marker loci may not be independent. When alleles at different marker loci are associated, the Bonferroni correction may lead to a conservative test, and hence a power loss. As an alternative, for tests of association that use family data, we propose a Monte Carlo procedure that provides a global assessment of significance. We examine the case of tightly linked markers with varying amounts of association between them. Using computer simulations, we study a family-based test for association (the transmission/disequilibrium test), and compare its power when either the Bonferroni or Monte Carlo procedure is used to determine significance. Our results show that when the alleles at different marker loci are not associated, using either procedure results in tests with similar power. However, when alleles at linked markers are associated, the test using the Monte Carlo procedure is more powerful than the test using the Bonferroni procedure. This proposed Monte Carlo procedure can be applied whenever it is suspected that markers examined have high amounts of association, or as a general approach to ensure appropriate significance levels and optimal power.

Alleles↗

PubMed2005

p-Value simulation for affected sib pair multiple testing.

A standard approach to calculation of critical values for affected sib pair multiple testing is based on: (a) fully informative markers, (b) Haldane map function assumptions leading to a Markov chain model for inheritance vectors, (c) central limit approximation to averages of sampled inheritance vectors leading to an Ornstein-Uhlenbeck process approximation, and (d) simple approximations to the maximum of such a process. Under these assumptions, assuming equispaced or close to equispaced markers, if the sample size is large, an approximation is available that is easy to calculate and performs well. However, for small sample sizes, a large number of markers, and for small p-values, there is good reason to be cautious about the use of the Gaussian approximation. We develop an algorithm for calculation of multiple testing p-values based on the standard Markov chain model, avoiding the use of Gaussian (large sample) approximation. We illustrate the use of this algorithm by demonstrating some inadequacies of the Gaussian approximation.

Algorithms↗

PubMed2005

Sample size calculation for simulation-based multiple-testing procedures.

In this article, we present a simple method to calculate sample size and power for a simulation-based multiple testing procedure which gives a sharper critical value than the standard Bonferroni method. The method is especially useful when several highly correlated test statistics are involved in a multiple-testing procedure. The formula for sample size calculation will be useful in designing clinical trials with multiple endpoints or correlated outcomes. We illustrate our method with a quality-of-life study for patients with early stage prostate cancer. Our method can also be used for comparing multiple independent groups.

Computer Simulation↗

PubMed2006

Analytical correction for multiple testing in admixture mapping.

Admixture mapping, using unrelated individuals from the admixture populations that result from recent mating between members of each parental population, is an efficient approach to localize disease-causing variants that differ in frequency between two or more historically separated populations. Recently, several methods have been proposed to test linkage between a susceptibility gene and a disease locus by using admixture-generated linkage disequilibrium (LD) for each of the genotyped markers. In a genome scan, admixture mapping usually tests 2,000 to 3,000 markers across the genome. Currently, either a very conservative Sidak (or Bonferroni) correction or a very time consuming simulation-based method is used to correct for the multiple tests and evaluate the overall p value. In this report, we propose a computationally efficient analytical approach for correction of the multiple tests and for calculating the overall p value for an admixture genome scan. Except for the Sidak (or Bonferroni) correction, our proposed method is the first analytical approach for correction of the multiple tests and for calculating the overall p value for a genome scan. Our simulation studies show that the proposed method gives correct overall type I error rates for genome scans in all cases, and is much more computationally efficient than simulation-based methods.

Computational Biology↗

PubMed2005

Multiple testing and data adaptive regression: an application to HIV-1 sequence data.

Analysis of viral strand sequence data and viral replication capacity could potentially lead to biological insights regarding the replication ability of HIV-1. Determining specific target codons on the viral strand will facilitate the manufacturing of target-specific antiretrovirals. Various algorithmic and analysis techniques can be applied to this application. In this paper, we apply two techniques to a data set consisting of 317 patients, each with 282 sequenced protease and reverse transcriptase codons. The first application is recently developed multiple testing procedures to find codons which have significant univariate associations with the replication capacity of the virus. A single-step multiple testing procedure (Pollard and van der Laan 2003) method was used to control the family wise error rate (FWER) at the five percent alpha level as well as the application of augmentation multiple testing procedures to control the generalized family wise error (gFWER) or the tail probability of the proportion of false positives (TPPFP). We also applied a data adaptive multiple regression algorithm to obtain a prediction of viral replication capacity based on an entire mutant/non-mutant sequence profile. This is a loss-based, cross-validated Deletion/Substitution/Addition regression algorithm (Sinisi and van der Laan 2004), which builds candidate estimators in the prediction of a univariate outcome by minimizing an empirical risk. These methods are two separate techniques with distinct goals used to analyze this structure of viral data.

Journal Article↗

PubMed2005

Empirical Bayes and resampling based multiple testing procedure controlling tail probability of the proportion of false positives.

Simultaneously testing a collection of null hypotheses about a data generating distribution based on a sample of independent and identically distributed observations is a fundamental and important statistical problem involving many applications. In this article we propose a new re-sampling based multiple testing procedure asymptotically controlling the probability that the proportion of false positives among the set of rejections exceeds q at level alpha, where q and alpha are user supplied numbers. The procedure involves 1) specifying a conditional distribution for a guessed set of true null hypotheses, given the data, which asymptotically is degenerate at the true set of null hypotheses, and 2) specifying a generally valid null distribution for the vector of test-statistics proposed in Pollard & van der Laan (2003), and generalized in our subsequent article Dudoit, van der Laan, & Pollard (2004), van der Laan, Dudoit, & Pollard (2004), and van der Laan, Dudoit, & Pollard (2004b). Ingredient 1) is established by fitting the empirical Bayes two component mixture model (Efron (2001b)) to the data to obtain an upper bound for marginal posterior probabilities of the null being true, given the data. We establish the finite sample rational behind our proposal, and prove that this new multiple testing procedure asymptotically controls the wished tail probability for the proportion of false positives under general data generating distributions. In addition, we provide simulation studies establishing that this method is generally more powerful in finite samples than our previously proposed augmentation multiple testing procedure (van der Laan, Dudoit, & Pollard (2004b)) and competing procedures from the literature. Finally, we illustrate our methodology with a data analysis.

Journal Article↗