SEARCH · PubMed Health

Results for “roc”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMed2000

Time-dependent ROC curves for censored survival data and a diagnostic marker.

ROC curves are a popular method for displaying sensitivity and specificity of a continuous diagnostic marker, X, for a binary disease variable, D. However, many disease outcomes are time dependent, D(t), and ROC curves that vary as a function of time may be more appropriate. A common example of a time-dependent variable is vital status, where D(t) = 1 if a patient has died prior to time t and zero otherwise. We propose summarizing the discrimination potential of a marker X, measured at baseline (t = 0), by calculating ROC curves for cumulative disease or death incidence by time t, which we denote as ROC(t). A typical complexity with survival data is that observations may be censored. Two ROC curve estimators are proposed that can accommodate censored data. A simple estimator is based on using the Kaplan-Meier estimator for each possible subset X > c. However, this estimator does not guarantee the necessary condition that sensitivity and specificity are monotone in X. An alternative estimator that does guarantee monotonicity is based on a nearest neighbor estimator for the bivariate distribution function of (X, T), where T represents survival time (Akritas, M. J., 1994, Annals of Statistics 22, 1299-1327). We present an example where ROC(t) is used to compare a standard and a modified flow cytometry measurement for predicting survival after detection of breast cancer and an example where the ROC(t) curve displays the impact of modifying eligibility criteria for sample size and power in HIV prevention trials.

Breast Neoplasms↗

PubMedDate not supplied

A general regression methodology for ROC curve estimation.

A method for applying generalized ordinal regression models to categorical rating data to estimate and analyze receiver operating characteristic (ROC) curves is presented. These models permit parsimonious adjustment of ROC curve parameters for relevant covariates through two regression equations that correspond to location and scale. Particular shapes of ROC curves are interpreted in relation to the kind of covariates included in the two regressions. The model is shown to be flexible because it is not restricted to the assumption of binormality that is commonly employed in smoothed ROC curve estimation, although the binormal model is one particular form of the more general model. The new method provides a mechanism for pinpointing the effect that interobserver variability has on the ROC curve. It also allows for the adjustment of ROC curves for temporal variation and case mix, and provides a way to assess the incremental diagnostic value of a test. The new methodology is recommended because it substantially improves the ability to assess diagnostic tests using ROC curves.

Breast Neoplasms↗

PubMedDate not supplied

Receiver operating characteristic (ROC) curve: practical review for radiologists.

The receiver operating characteristic (ROC) curve, which is defined as a plot of test sensitivity as they coordinate versus its 1-specificity or false positive rate (FPR) as the x coordinate, is an effective method of evaluating the performance of diagnostic tests. The purpose of this article is to provide a nonmathematical introduction to ROC analysis. Important concepts involved in the correct use and interpretation of this analysis, such as smooth and empirical ROC curves, parametric and nonparametric methods, the area under the ROC curve and its 95% confidence interval, the sensitivity at a particular FPR, and the use of a partial area under the ROC curve are discussed. Various considerations concerning the collection of data in radiological ROC studies are briefly discussed. An introduction to the software frequently used for performing ROC analyses is also presented.

Area Under Curve↗

PubMed1988

Analysis of transcription termination signals in the nin region of bacteriophage lambda: the roc deletion.

Deletions in the region, nin, between the P and Q genes of phage lambda remove a portion of the phage genome that includes signals for termination of transcription. These deletions were selected because they permit growth of lambda derivatives defective in the N-mediated transcription antitermination system; i.e., the deletions confer N independence (nin). Thus nin phages (e.g., lambda nin5) grow in most Escherichia coli nus mutants. The nus genes encode functions necessary for N action. We report the isolation of a deletion in the nin region delta roc that confers a partially N-independent phenotype; lambda derivatives with delta roc can grow under normally nonpermissive conditions, 32 degrees C, in a host with the rpoB-nusC60 mutation. The roc deletion also partially suppresses the inhibitory effects of other nus mutations at higher temperatures. Delta roc, which extends from base pairs 41883 to 43825, overlaps the nin5 deletion, which extend from base pairs 40501 to 43306. Unlike the nin5 deletion, the sequences deleted by delta roc do not include a stem-loop structure, tR2, previously shown to have terminator activity. Using promoter and terminator testor vectors, we demonstrate that a 2,400-base-pair fragment that includes the wild-type roc region but excludes tR2 has terminator activity. Thus, delta roc permits a functional division of the transcription termination signals in the nin region.

Bacteriophage lambda↗

PubMed1985

An evaluation of methods for estimating the area under the receiver operating characteristic (ROC) curve.

The area under the receiver operating characteristic (ROC) curve serves as one means for evaluating the performance of diagnostic and predictive test systems. The most commonly used method for estimating the area under an ROC curve utilizes the maximum-likelihood-estimation technique, and a nonparametric method to calculate the area under an ROC curve was recently described. We compared the performance of these two methods. The results for the area under the ROC curve and the standard error of the estimate as calculated by each of the two methods exhibited high correlation. Generally, the nonparametric method yields lower area estimates than the maximum-likelihood-estimation technique. However, these differences generally were small, particularly with ROC curves derived from five or more cutoff points. Consistent results of hypothesis testing of the significance of differences between two ROC curves will be similar, regardless of which method is used, as long as one uses the same estimation technique on the two curves and as long as the two ROC curves being compared are of similar shape.

Computers↗

PubMed1997

Sample size determination for diagnostic accuracy studies involving binormal ROC curve indices.

Receiver operating characteristic (ROC) curves and their associated indices are valuable tools for the assessment of the accuracy of diagnostic tests. The area under the ROC curve is a popular summary measure of the accuracy of a test. The full area under the ROC curve, however, has been criticized because it gives equal weight to all false positive error rates. Alternative indices include the area under the ROC curve in a particular range of false positive rates ('partial' area) and the sensitivity of the test for a single fixed false positive rate (FPR). We present a unified approach for computing sample size for binormal ROC curves and their indices. Our method uses Taylor series expansions to derive approximate large-sample estimates of the variance and covariance of binormal ROC curve parameters. Several examples from diagnostic radiology illustrate the proposed method.

Diagnostic Tests, Routine↗

PubMed2002

A non-parametric method for the comparison of partial areas under ROC curves and its application to large health care data sets.

The receiver operating characteristic (ROC) curve is a statistical tool for evaluating the accuracy of diagnostic tests. Investigators often compare the validity of two tests based on the estimated areas under the respective ROC curves. However, the traditional way of comparing entire areas under two ROC curves is not sensitive when two ROC curves cross each other. Also, there are some cutpoints on the ROC curves that are not considered in practice because their corresponding sensitivities or specificities are unacceptable. For the purpose of comparing the partial area under the curve (AUC) within a specific range of specificity for two correlated ROC curves, a non-parametric method based on Mann-Whitney U-statistics has been developed. The estimation of AUC along with its estimated variance and covariance is simplified by a method of grouping the observations according to their cutpoint values. The method is used to evaluate alternative logistic regression models that predict whether a subject has incident breast cancer based on information in Medicare claims data.

Area Under Curve↗

PubMed1996

Signal-detection, threshold, and dual-process models of recognition memory: ROCs and conscious recollection.

Threshold- and signal-detection-based models have dominated theorizing about recognition memory. Building upon these theoretical frameworks, we have argued for a dual-process model in which conscious recollection (a threshold process) and familiarity (a signal-detection process) contribute to memory performance. In the current paper we assessed several memory models by examining the effects of levels of processing and the number of presentations on recognition memory receiver operating characteristics (ROCs). In general, when the ROCs were plotted in probability space they exhibited an inverted U shape; however, when they were plotted in z space they exhibited a U shape. An examination of the ROCs showed that the dual-process model could account for the observed ROCs, but that models based solely on either threshold or signal-detection processes failed to provide a sufficient account of the data. Furthermore, an examination of subjects' introspective reports using the remember/know procedure showed that subjects were aware of recollection and familiarity and were able to consistently report on their occurrence. The remember/know data were used to accurately predict the shapes of the ROCs, and estimates of recollection and familiarity derived from the ROC data mirrored the subjective reports of these processes.

Adult↗

PubMedDate not supplied

Receiver operating characteristic (ROC) analysis of neurons in the cat's lateral geniculate nucleus during tonic and burst response mode.

Relay cells of the lateral geniculate nucleus respond to visual stimuli in one of two modes: burst and tonic. The burst mode depends on the activation of a voltage-dependent, Ca2+ conductance underlying the low threshold spike. This conductance is inactivated at depolarized membrane potentials, but when activated from hyperpolarized levels, it leads to a large, triangular, nearly all-or-none depolarization. Typically, riding its crest is a high-frequency barrage of action potentials. Low threshold spikes thus provide a nonlinear amplification allowing hyperpolarized relay neurons to respond to depolarizing inputs, including retinal EPSPs. In contrast, the tonic mode is characterized by a steady stream of unitary action potentials that more linearly reflects the visual stimulus. In this study, we tested possible differences in detection between response modes of 103 geniculate neurons by constructing receiver operating characteristic (ROC) curves for responses to visual stimuli (drifting sine-wave gratings and flashing spots). Detectability was determined from the ROC curves by computing the area under each curve, known as the ROC area. Most cells switched between modes during recording, evidently due to small shifts in membrane potential that affected the activation state of the low threshold spike. We found that the more often a cell responded in burst mode, the larger its ROC area. This was true for responses to optimal and nonoptimal visual stimuli, the latter including nonoptimal spatial frequencies and low stimulus contrasts. The larger ROC areas associated with burst mode were due to a reduced spontaneous activity and roughly equivalent level of visually evoked response when compared to tonic mode. We performed a within-cell analysis on a subset of 22 cells that switched modes during recording. Every cell, whether tested with a low contrast or high contrast visual stimulus exhibited a larger ROC area during its burst response mode than during its tonic mode. We conclude that burst responses better support signal detection than do tonic responses. Thus, burst responses, while less linear and perhaps less useful in providing a detailed analysis of visual stimuli, improve target detection. The tonic mode, with its more linear response, seems better suited for signal analysis rather than signal detection.

Animals↗

PubMed1997

Generating ROC curves for artificial neural networks.

Receiver operating characteristic (ROC) analysis is an established method of measuring diagnostic performance in medical imaging studies. Traditionally, artificial neural networks (ANN's) have been applied as a classifier to find one "best" detection rate. Recently researchers have begun to report ROC curve results for ANN classifiers. The current standard method of generating ROC curves for an ANN is to vary the output node threshold for classification. In this work, we propose a different technique for generating ROC curves for a two-class ANN classifier. We show that this new technique generates better ROC curves in the sense of having greater area under the ROC curve (AUC), and in the sense of being composed of a better distribution of operating points.

Diagnostic Imaging↗

PubMed2000

Regression models for convex ROC curves.

The performance of a diagnostic test is summarized by its receiver operating characteristic (ROC) curve. Under quite natural assumptions about the latent variable underlying the test, the ROC curve is convex. Empirical data on a test's performance often comes in the form of observed true positive and false positive relative frequencies under varying conditions. This paper describes a family of regression models for analyzing such data. The underlying ROC curves are specified by a quality parameter delta and a shape parameter mu and are guaranteed to be convex provided delta > 1. Both the position along the ROC curve and the quality parameter delta are modeled linearly with covariates at the level of the individual. The shape parameter mu enters the model through the link functions log(p mu) - log(1 - p mu) of a binomial regression and is estimated either by search or from an appropriate constructed variate. One simple application is to the meta-analysis of independent studies of the same diagnostic test, illustrated on some data of Moses, Shapiro, and Littenberg (1993). A second application, to so-called vigilance data, is given, where ROC curves differ across subjects and modeling of the position along the ROC curve is of primary interest.

Biometry↗

PubMed2005

A tutorial on the use of ROC analysis for computer-aided diagnostic systems.

The application of the receiver operating characteristic (ROC) curve for computer-aided diagnostic systems is reviewed. A statistical framework is presented and different methods of evaluating the classification performance of computer-aided diagnostic systems, and, in particular, systems for ultrasonic tissue characterization, are derived. Most classifiers that are used today are dependent on a separation threshold, which can be chosen freely in many cases. The separation threshold separates the range of output values of the classification system into different target groups, thus conducting the actual classification process. In the first part of this paper, threshold specific performance measures, e.g., sensitivity and specificity, are presented. In the second part, a threshold-independent performance measure, the area under the ROC curve, is reviewed. Only the use of separation threshold-independent performance measures provides classification results that are overall representative for computer-aided diagnostic systems. The following text was motivated by the lack of a complete and definite discussion of the underlying subject in available textbooks, references and publications. Most manuscripts published so far address the theme of performance evaluation using ROC analysis in a manner too general to be practical for everyday use in the development of computer-aided diagnostic systems. Nowadays, the user of computer-aided diagnostic systems typically handles huge amounts of numerical data, not always distributed normally. Many assumptions made in more or less theoretical works on ROC analysis are no longer valid for real-life data. The paper aims at closing the gap between theoretical works and real-life data. The review provides the interested scientist with information needed to conduct ROC analysis and to integrate algorithms performing ROC analysis into classification systems while understanding the basic principles of classification.

Diagnosis, Computer-Assisted↗

PubMed1995

Distinction between homozygous and heterozygous subjects with hereditary haemochromatosis using iron status markers and receiver operating characteristic (ROC) analysis.

The purpose of the present study was: 1) to evaluate which of the four iron status markers (serum iron, serum transferrin, serum transferrin saturation, serum ferritin) displayed the highest discriminatory potential in the distinction between homozygous patients with hereditary haemochromatosis and heterozygous relatives, 2) to suggest optimum cut-off values for these iron status markers, and 3) to demonstrate how these cut-off values change if the expected utility from a correct diagnosis is incorporated into the analysis. The patients and relatives were found by a nation-wide epidemiological survey. The study population consisted of 162 patients with clinically overt hereditary haemochromatosis and 84 asymptomatic heterozygous relatives. The statistical evaluation was performed using receiver operating characteristic (ROC) curve analysis. The diagnostic power of the iron markers is expressed as the area under the ROC curve. The optimum cut-off value is at the point where the slope of the ROC curve is equal to one. Changes in the optimum cut-off value at varying expected utility from a correct classification was estimated by changing the scaling of the ROC diagram. Serum iron and serum transferrin had the smallest area under the ROC curve, and were both unsuitable as discriminators. Near complete discrimination was obtained with serum transferrin saturation and serum ferritin concentrations, displaying the largest area under the ROC curve (0.991 and 0.998). The optimum threshold value for transferrin saturation was 61%, and for serum ferritin concentration 800 micrograms/l. The transferrin saturation level reflects the presence of the haemochromatosis allele, whereas the serum ferritin concentration indicates the degree of iron overload.(ABSTRACT TRUNCATED AT 250 WORDS)

Adult↗

PubMed2005

ROCPLOT: a generic software tool for ROC analysis and the validation of predictive methods.

UNLABELLED: Receiver operating characteristic (ROC) analysis is a powerful and widely used technique for assessing predictive methods, yet there are no generic, open-source software tools for this that are freely available. Our ROCPLOT program performs ROC analysis on one or more files of search results (hits) and generates the following: (i) ROC values, giving a convenient numerical measure of method sensitivity and specificity; (ii) ROC plots graphically displaying sensitivity and specificity; (iii) classification plots to aid interpretation of the ROC plots and values; and (iv) a bar chart of the distribution of ROC values. ROCPLOT is generic and flexible: data in multiple hits files can be processed in series or parallel, allowing the results of multiple predictions to be viewed side-by-side or combined. AVAILABILITY: ROCPLOT is freely available for download as part of the European Molecular Biology Open Software Suite, EMBOSS (http://emboss.sourceforge.net/apps/rocplot.html).

Computer Graphics↗

PubMed1990

[The application of medical decision making to the mass screening: the basic principles on ROC analysis].

The basic principles on receiver-operating characteristic (ROC) analysis were discussed. ROC curves showed the discriminative ability of a test by the position of the full curve in a graph plotting the relation between the true positive rate (TPR) and the false positive rate (FPR) over a wide range of cut-off points of a test. The increase in the area under the ROC curve, or the shift of the curve upward and to the left in the diagram means that the test has better discriminative ability. A manual was given to conduct the ROC analysis with special reference to calculation of TPR, FPR, and the area under the ROC curve. Also discussed was the method to decide the best or optimal cut-off point as the positivity criterion of a test, based on the ROC analysis. Attention was paid to balance the risk of false negatives and false positives. We made an equation to decide the best cut-off point, which showed us the variables to be considered in the analysis of cut-off problems: the prevalence of disease and the outcome associated with each state classified by the test, i.e., true positive, false positive, true negative, and false negative.

Decision Support Techniques↗

PubMed1999

"Proper" Binormal ROC Curves: Theory and Maximum-Likelihood Estimation.

The conventional binormal model, which assumes that a pair of latent normal decision-variable distributions underlies ROC data, has been used successfully for many years to fit smooth ROC curves. However, if the conventional binormal model is used for small data sets or ordinal-category data with poorly allocated category boundaries, a "hook" in the fitted ROC may be evident near the upper-right or lower-left corner of the unit square. To overcome this curve-fitting artifact, we developed a "proper" binormal model and a new algorithm for maximum-likelihood (ML) estimation of the corresponding ROC curves. Extensive simulation studies have shown the algorithm to be highly reliable. ML estimates of the proper and conventional binormal ROC curves are virtually identical when the conventional binormal ROC shows no "hook," but the proper binormal curves have monotonic slope for all data sets, including those for which the conventional model produces degenerate fits. Copyright 1999 Academic Press.

Journal Article↗

PubMed1986

ROC methodology in radiologic imaging.

If the performance of a diagnostic imaging system is to be evaluated objectively and meaningfully, one must compare radiologists' image-based diagnoses with actual states of disease and health in a way that distinguishes between the inherent diagnostic capacity of the radiologists' interpretations of the images, and any tendencies to "under-read" or "over-read". ROC methodology provides the only known basis for distinguishing between these two aspects of diagnostic performance. After identifying the fundamental issues that motivate ROC analysis, this article develops ROC concepts in an intuitive way. The requirements of a valid ROC study and practical techniques for ROC data collection and data analysis are sketched briefly. A survey of the radiologic literature indicates the broad variety of evaluation studies in which ROC analysis has been employed.

Biometry↗

PubMed1997

Receiver operating characteristic (ROC) curve analysis of transrectal ultrasonographic prostate cancer detection.

PURPOSE: To provide an overview of receiver operating characteristics (ROC) curve theory and create an ROC curve describing transrectal ultrasound (TRUS) detection of prostate cancer (PCa). METHODS: One hundred thirty-two patients with either an abnormal digital rectal exam (DRE) or a prostate-specific antigen (PSA) above 4 ng/ml or both underwent TRUS and biopsy of the peripheral zone (PZ) of each quadrant. ROC software was used to create an ROC curve. RESULTS: One hundred seventy-nine of 528 quadrants (34%) harbored PCa. The performance of TRUS in detection of PCa as defined by the area below the ROC is 0,809. CONCLUSIONS: Future investigators are encouraged to employ ROC analysis of TRUS to permit more meaningful comparisons of performance.

Journal Article↗