SEARCH · PubMed Health

Results for “ROC Curve”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMed2002

A non-parametric method for the comparison of partial areas under ROC curves and its application to large health care data sets.

The receiver operating characteristic (ROC) curve is a statistical tool for evaluating the accuracy of diagnostic tests. Investigators often compare the validity of two tests based on the estimated areas under the respective ROC curves. However, the traditional way of comparing entire areas under two ROC curves is not sensitive when two ROC curves cross each other. Also, there are some cutpoints on the ROC curves that are not considered in practice because their corresponding sensitivities or specificities are unacceptable. For the purpose of comparing the partial area under the curve (AUC) within a specific range of specificity for two correlated ROC curves, a non-parametric method based on Mann-Whitney U-statistics has been developed. The estimation of AUC along with its estimated variance and covariance is simplified by a method of grouping the observations according to their cutpoint values. The method is used to evaluate alternative logistic regression models that predict whether a subject has incident breast cancer based on information in Medicare claims data.

Area Under Curve↗

PubMed1985

A Visicalc program for estimating the area under a receiver operating characteristic (ROC) curve.

The area under the ROC curve interests us as a method for analyzing discrimination or detectability. One can assess a diagnostic test or probability assessor with respect to its degree of discrimination. The area under the ROC curve gives us the probability of correctly identifying abnormal from normal in a forced-choice, two-alternative problem. Previous methods used for calculating the area involved maximum likelihood estimation on a mainframe or minicomputer. This paper demonstrates an adaptation of a recently published nonparametric method for estimating the area. This adaptation takes advantage of electronic spreadsheet software and may be used on most (if not all) microcomputers. The paper develops the construction of the program needed for the necessary calculations.

Computers↗

PubMed2000

Regression models for convex ROC curves.

The performance of a diagnostic test is summarized by its receiver operating characteristic (ROC) curve. Under quite natural assumptions about the latent variable underlying the test, the ROC curve is convex. Empirical data on a test's performance often comes in the form of observed true positive and false positive relative frequencies under varying conditions. This paper describes a family of regression models for analyzing such data. The underlying ROC curves are specified by a quality parameter delta and a shape parameter mu and are guaranteed to be convex provided delta > 1. Both the position along the ROC curve and the quality parameter delta are modeled linearly with covariates at the level of the individual. The shape parameter mu enters the model through the link functions log(p mu) - log(1 - p mu) of a binomial regression and is estimated either by search or from an appropriate constructed variate. One simple application is to the meta-analysis of independent studies of the same diagnostic test, illustrated on some data of Moses, Shapiro, and Littenberg (1993). A second application, to so-called vigilance data, is given, where ROC curves differ across subjects and modeling of the position along the ROC curve is of primary interest.

Biometry↗

PubMed2004

ROC curves in clinical chemistry: uses, misuses, and possible solutions.

BACKGROUND: ROC curves have become the standard for describing and comparing the accuracy of diagnostic tests. Not surprisingly, ROC curves are used often by clinical chemists. Our aims were to observe how the accuracy of clinical laboratory diagnostic tests is assessed, compared, and reported in the literature; to identify common problems with the use of ROC curves; and to offer some possible solutions. METHODS: We reviewed every original work using ROC curves and published in Clinical Chemistry in 2001 or 2002. For each article we recorded phase of the research, prospective or retrospective design, sample size, presence/absence of confidence intervals (CIs), nature of the statistical analysis, and major analysis problems. RESULTS: Of 58 articles, 31% were phase I (exploratory), 50% were phase II (challenge), and 19% were phase III (advanced) studies. The studies increased in sample size from phase I to III and showed a progression in the use of prospective designs. Most phase I studies were powered to assess diagnostic tests with ROC areas >/=0.70. Thirty-eight percent of studies failed to include CIs for diagnostic test accuracy or the CIs were constructed inappropriately. Thirty-three percent of studies provided insufficient analysis for comparing diagnostic tests. Other problems included dichotomization of the gold standard scale and inappropriate analysis of the equivalence of two diagnostic tests. CONCLUSION: We identify available software and make some suggestions for sample size determination, testing for equivalence in diagnostic accuracy, and alternatives to a dichotomous classification of a continuous-scale gold standard. More methodologic research is needed in areas specific to clinical chemistry.

Chemistry, Clinical↗

PubMedDate not supplied

An exponential model used for optimal threshold selection on ROC curves.

A two-parameter exponential equation for modeling a receiver operating characteristic (ROC) curve is presented, where the area under the curve is a simple function of one of the parameters. The model makes no distributional assumptions about the underlying normal and abnormal patient populations or about the shape of the resulting ROC curves. In a computer simulation of 75 ROC curves, the model provides a fit equivalent to the maximum likelihood estimate method commonly used for ROC curve fitting. Similar results are obtained using the model to fit ROC curve data from the literature. The model's equation calculates the true-positive ratio as a function of the false-positive ratio, and has a first derivative that is useful for finding the optimal decision threshold for a diagnostic testing procedure. In particular, the model is useful in a computer program for finding jointly optimal thresholds for multiple sequential tests.

Computer Simulation↗

PubMedDate not supplied

A general regression methodology for ROC curve estimation.

A method for applying generalized ordinal regression models to categorical rating data to estimate and analyze receiver operating characteristic (ROC) curves is presented. These models permit parsimonious adjustment of ROC curve parameters for relevant covariates through two regression equations that correspond to location and scale. Particular shapes of ROC curves are interpreted in relation to the kind of covariates included in the two regressions. The model is shown to be flexible because it is not restricted to the assumption of binormality that is commonly employed in smoothed ROC curve estimation, although the binormal model is one particular form of the more general model. The new method provides a mechanism for pinpointing the effect that interobserver variability has on the ROC curve. It also allows for the adjustment of ROC curves for temporal variation and case mix, and provides a way to assess the incremental diagnostic value of a test. The new methodology is recommended because it substantially improves the ability to assess diagnostic tests using ROC curves.

Breast Neoplasms↗

PubMed1990

On the shape of energy-detection ROC curves.

A numerical study shows that energy-detection ROC curves based on chi-squared distributions may be accurately represented by binormal receiver operating characteristic (ROC) curves. This allows the detector accuracy and the ROC shape (asymmetry) to be expressed simply in terms of the distribution parameters. An established approximate formula for the shape of energy-detection ROC curves is shown to be inaccurate. An improved approximately is presented.

Attention↗

PubMed1995

ROC curves with multiple marker measurements.

Properties of receiver operating characteristic (ROC) curves are explored for markers that are measured repeatedly, through space or time, for each subject. The true underlying response, positive or negative, of each subject is assumed to be constant across marker measurements, and is determined from assessment of some "gold standard." A marker-based test is considered positive when at least one of a subject's marker values exceeds a designated cutoff. If subjects with positive and negative underlying responses differ in the number of marker measurements per subject, the ROC curve for a noninformative marker is bowed above or below the diagonal line representing the "null" curve for a marker that is measured just once per subject. If subjects with negative responses tend to have more measurements than those with positive responses, the ROC curve for even an informative marker may lie beneath the curve for the same marker measured once per subject. The form of the ROC curve for a marker used in this way is strongly influenced by the strength of the correlation of measurements within subjects.

Biomarkers↗

PubMedDate not supplied

Goodness-of-fit issues in ROC curve estimation.

Zhou recently considered goodness-of-fit (GOF) testing for receiver operating characteristic (ROC) curves estimated by applying the binormal and other bidistributional models to rating method data. He interpreted significant GOF tests as evidence that different decision thresholds were applied to diseased and nondiseased subjects and concluded that, in such circumstances, an ROC curve does not exist. In this article the author demonstrates that the GOF test accommodates many alternative hypotheses and that a significant test result need not be equated with an interaction between disease status and decision criteria or with non-existence of an ROC curve. He develops a new family of ROC curves based on a fully parameterized bidistributional model. The family includes the binormal ROC curve, but generalizes its structure by signifying all identifiable deviations in parameters of the latent distributions that define the model. The family provides a unified framework of alternative models to the binormal assumption and of alternative hypotheses to the GOF test for that assumption.

Binomial Distribution↗

PubMedDate not supplied

Testing an underlying assumption on a ROC curve based on rating data.

The receiver operating characteristic (ROC) curve is widely accepted as the measurement that best represents the accuracy of a diagnostic system that consists of a diagnostic imaging modality and a reader who is interpreting results from the modality. However, there is one implicit assumption in a ROC curve: that the cutoff points to declare a positive test for decision variables of a diseased patient and a nondiseased patient are the same. If this assumption is violated, then a ROC curve does not exist. The author develops a statistical procedure to test this assumption. To demonstrate, he applies the method to a ROC data set and shows that the assumption of the equal cutoff points could be rejected.

Data Interpretation, Statistical↗

PubMed1985

The use of ROC curves in histopathologic decision making.

The applicability of receiver operator characteristic (ROC) curves to histopathology is presented in three examples, along with a basic discussion of some properties of ROC curves. One major application is in the attempt to define stages in diseases that show a continuous spectrum of histologic patterns, in which the uncertainty of boundary points and the overlap of features makes such definition difficult. Construction and analysis of ROC curves may help to identify the features with the greatest utility, as, for example, in the grading of mucinous carcinomas of the ovary. ROC curves can also be used to assess diagnostic differences between histopathologists, whether they are using different criteria or the same criteria but with different weightings, as, for example, in cervical premalignancy or borderline ovarian tumors.

Adenocarcinoma, Mucinous↗

PubMedDate not supplied

Determining a range of false-positive rates for which ROC curves differ.

Many indices have been proposed for summarizing the information contained in the ROC curve. When comparing two ROC curves, though, there are times when global summary measures are either not optimal or not appropriate. The author presents a method for directly comparing true-positive rates for two diagnostic, screening or prognostic tools, determining over what range of false-positive values the tests differ. The method is applicable for independent or dependent samples. An example concerning gallium citrate imaging is presented, as well as an example using a prognostic index for severity of illness in the ICU. The range of false-positive rates for which the ROC curves differ is determined for each example.

Bacterial Infections↗

PubMed2000

ROC curve estimation when covariates affect the verification process.

A receiver operating characteristic (ROC) curve is commonly used to measure the accuracy of a medical test. It is a plot of the true positive fraction (sensitivity) against the false positive fraction (1-specificity) for increasingly stringent positivity criterion. Bias can occur in estimation of an ROC curve if only some of the tested patients are selected for disease verification and if analysis is restricted only to the verified cases. This bias is known as verification bias. In this paper, we address the problem of correcting for verification bias in estimation of an ROC curve when the verification process and efficacy of the diagnostic test depend on covariates. Our method applies the EM algorithm to ordinal regression models to derive ML estimates for ROC curves as a function of covariates, adjusted for covariates affecting the likelihood of being verified. Asymptotic variance estimates are obtained using the observed information matrix of the observed data. These estimates are derived under the missing-at-random assumption, which means that selection for disease verification depends only on the observed data, i.e., the test result and the observed covariates. We also address the issues of model selection and model checking. Finally, we illustrate the proposed method on data from a two-phase study of dementia disorders, where selection for verification depends on the screening test result and age.

Aged↗

PubMedDate not supplied

Statistical comparison of ROC curves from multiple readers.

Receiver operating characteristic (ROC) analysis is the commonly accepted method for comparing diagnostic imaging systems. In general, ROC studies are designed in such a way that multiple readers read the same images and each image is presented by means of two different imaging systems. Statistical methods for the comparison of the ROC curves from one reader have been developed, but extension of these methods to multiple readers is not straightforward. A new method of analysis is presented for the comparison of ROC curves from multiple readers. This method includes a non-parametric estimation of the variances and covariances between the various areas under the curves. The method described is more appropriate than the paired t test, because it also takes the case-sample variation into account.

Breast Neoplasms↗

PubMedDate not supplied

Signal detectability: the use of ROC curves and their analyses.

Issues related to ROC curves are addressed. The original article on the subject by Lee Lusted, describing the "state of the art" 20 years ago, is reviewed. The concepts that Lusted addressed are then expanded, suggesting the current state of the art. New issues that have arisen with regard to ROC curves and their use in medicine are addressed. Finally, potential areas for future investigation are suggested.

Diagnosis, Computer-Assisted↗

PubMed1990

[ROC-curve analysis. A statistical method for the evaluation of diagnostic tests].

Receiver operating characteristic (ROC)-curves are a statistical method which may be employed inter alii for assessing a diagnostic test. These have been employed particularly in radiology but have also been employed for assessing laboratory tests and obstetric estiamation scales. The method is based on an analysis of nosological probabilities for varying limits of decision thresholds. ROC-curves may be employed for data on ratio/interval scales and data on rank scales. Parametric and non-parametric methods are available for obtaining a single quantitative measurement for the entire ROC-curve and to carry out significance tests between several ROC-curves.

Diagnosis↗

PubMed1986

The use of relative operating characteristic (ROC) curves in test performance evaluation.

Relative (or receiver) operating characteristic (ROC) curves are a graphical representation of the relationship between sensitivity and specificity of a laboratory test over all possible diagnostic cutoff values. Laboratory medicine has been slow to adopt the use of ROC curves for the analysis of diagnostic test performance. In this tutorial, we discuss the advantages and limitations of the ROC curve for clinical decision making in laboratory medicine. We demonstrate the construction and statistical uses of ROC analysis, review its published applications in clinical pathology, and comment on its role in the decision analytic framework in laboratory medicine.

Clinical Laboratory Techniques↗

PubMed2005

A permutation test sensitive to differences in areas for comparing ROC curves from a paired design.

The area under the receiver operating characteristic (ROC) curve (AUC) is a widely accepted summary index of the overall performance of diagnostic procedures and the difference between AUCs is often used when comparing two diagnostic systems. We developed an exact non-parametric statistical procedure for comparing two ROC curves in paired design settings. The test which is based on all permutations of the subject specific rank ratings is formally a test for equality of ROC curves that is sensitive to the alternatives of AUC difference. The operating characteristics of the proposed test were evaluated using extensive simulations over a wide range of parameters. The proposed procedure can be easily implemented in experimental ROC data sets. For small samples and for underlying parameters that are common in experimental studies in diagnostic imaging the test possesses good operating characteristics and is more powerful than the conventional non-parametric procedure for AUC comparisons. We also derived an asymptotic version of the test which uses an exact estimate of the variance in the permutation space and provides a good approximation even when the sample sizes are small. This asymptotic procedure is a simple and precise approximation to the exact test and is useful for large sample sizes where the exact test may be computationally burdensome.

Biometry↗