Systems validation: application to statistical programs.

Rickey E Carter

doi:10.1186/1471-2288-5-3

Home/Archive/PubMed

PubMed · 15649319

Systems validation: application to statistical programs.

Rickey E Carter

View original source ↗Export metadata ↓

Abstract

BACKGROUND: In 2003, the United States Food and Drug Administration (FDA) released a guidance document on the scope of "Part 11" enforcement. In this guidance document, the FDA indicates an expectation of a risk-based approach to determining which systems should undergo validation. Since statistical programs manage and manipulate raw data, their implementation should be critically reviewed to determine whether or not they should undergo validation. However, the concepts of validation are not often discussed in biostatistics curriculum. DISCUSSION: This paper summarizes a "Plan, Do, Say" approach to validation that can be incorporated into statistical training so that biostatisticians can understand and implement validation principles in their research. SUMMARY: Validation is a process that requires dedicated attention. The process of validation can be easily understood in the context of the scientific method.

Explore related subjects

Clinical Trials as Topic Computer Systems Guidelines as Topic Humans Software Validation Systems Analysis United States United States Food and Drug Administration Journal Article Research Support, N.I.H., Extramural Research Support, U.S. Gov't, P.H.S.

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Rickey E Carter. 2005-01-13. Systems validation: application to statistical programs.. https://doi.org/10.1186/1471-2288-5-3

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2007

Performance of adaptive sample size adjustment with respect to stopping criteria and time of interim analysis.

The benefit of adjusting the sample size in clinical trials on the basis of treatment effects observed in interim analysis has been the subject of several recent papers. Different conclusions were drawn about the usefulness of this approach for gaining power or saving sample size, because of differences in trial design and setting. We examined the benefit of sample size adjustment in relation to trial design parameters such as 'time of interim analysis' and 'choice of stopping criteria'. We compared the adaptive weighted inverse normal method with classical group sequential methods for the most common and for optimal stopping criteria in early, half-time and late interim analyses. We found that reacting to interim data might significantly reduce average sample size in some situations, while classical approaches can out-perform the adaptive designs under other circumstances. We characterized these situations with respect to time of interim analysis and choice of stopping criteria.

Clinical Trials as Topic↗

PubMed2007

Appropriateness of some resampling-based inference procedures for assessing performance of prognostic classifiers derived from microarray data.

The goal of many gene-expression microarray profiling clinical studies is to develop a multivariate classifier to predict patient disease outcome from a gene-expression profile measured on some biological specimen from the patient. Often some preliminary validation of the predictive power of a profile-based classifier is carried out using the same data set that was used to derive the classifier. Techniques such as cross-validation or bootstrapping can be used in this setting to assess predictive power, and if applied correctly, can result in a less biased estimate of predictive accuracy of a classifier. However, some investigators have attempted to apply standard statistical inference procedures to assess the statistical significance of associations between true and cross-validated predicted outcomes. We demonstrate in this paper that naïve application of standard statistical inference procedures to these measures of association under null situations can result in greatly inflated testing type I error rates. Under alternatives of small to moderate associations, confidence interval coverage probabilities may be too low, although for very large associations coverage probabilities approach their intended values. Our results suggest that caution should be exercised in interpreting some of the claims of exceptional prognostic classifier performance that have been reported in prominent biomedical journals in the past few years.

Clinical Trials as Topic↗

PubMed2006

Some design issues of strata-matched non-randomized studies with survival outcomes.

Non-randomized studies for the evaluation of a medical intervention are useful for quantitative hypothesis generation before the initiation of a randomized trial and also when randomized clinical trials are difficult to conduct. A strata-matched non-randomized design is often utilized where subjects treated by a test intervention are matched to a fixed number of subjects treated by a standard intervention within covariate based strata. In this paper, we consider the issue of sample size calculation for this design. Based on the asymptotic formula for the power of a stratified log-rank test, we derive a formula to calculate the minimum number of subjects in the test intervention group that is required to detect a given relative risk between the test and standard interventions. When this minimum number of subjects in the test intervention group is available, an equation is also derived to find the multiple that determines the number of subjects in the standard intervention group within each stratum. The methodology developed is applied to two illustrative examples in gastric cancer and sarcoma.

Clinical Trials as Topic↗