SEARCH · PubMed Health

Results for “roc”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6Linked to original sources

PubMed1991

ROC curves and the binormal assumption.

Previous articles in this series have described how receiver operating characteristic (ROC) graphs provide comprehensive graphic representations of the diagnostic performance of non-binary tests and have explained how one constructs "trapezoidal" ROC graphs in which discrete cutoff points are plotted and connected with line segments. In this article, we describe a set of mathematical assumptions that permit the generation of a continuous, smooth ROC curve for a given diagnostic test. These assumptions permit us to characterize a test's performance using a small number of parameters and also to explore properties of diagnostic tests. In this article, we describe a set of mathematical assumptions that can be used to link receiver operating characteristic (ROC) curves to the underlying distribution of values of the diagnostic variable being measured. We will illustrate these assumptions using a diagnostic test that distinguishes alcohol abusers from normal consumers of alcohol and abstainers.

Alcoholism↗

PubMedDate not supplied

Comparing three-class diagnostic tests by three-way ROC analysis.

Three-way ROC surfaces are based on a generalization of dichotomous ROC analysis to three-class diagnostic tests. The discriminatory power of three-class diagnostic tests is measured by the volume under the ROC surface. This measure can be given a probabilistic interpretation similar to the equivalence of the c-index to the area under the ROC curve. This article presents a method to calculate nonparametric estimates of the variance of the volume under the surface using Mann-Whitney U statistics. As a simple extension of this result, it is possible to calculate covariance estimates for the volume under the surface. This allows the statistical comparison of two tests used for diagnostic tasks with three possible outcomes. The formulas derived are validated on synthetic data and applied to a three-class data set of pigmented skin lesions. It is shown that a neural network algorithm trained on clinical data and lesion features performs better than one trained on only the lesion features.

Humans↗

PubMedDate not supplied

Meta-analysis of ROC curves.

The authors present a method to combine several independent studies of the same (continuous or semiquantitative) diagnostic test, where each study reports a complete ROC curve; a plot of the true-positive rate or sensitivity against the false-positive rate or one minus the specificity. The result of the analysis is a pooled ROC curve, with a confidence band, as opposed to earlier proposals that result in a pooled area under the ROC curve. The analysis is based on a two-parameter model for the ROC curve that can be estimated for each individual curve. The parameters are then pooled with a bivariate random-effects meta-analytic method, and a curve can be drawn from the pooled parameters. The authors propose to use a model that specifies a linear relation between the logistic transformations of sensitivity and one minus specificity. Specifically, they define V = In(sensitivity/(1 - sensitivity)) and U = In((1 - specificity)/specificity), and then D = V - U, S = V + U. The model is defined as D = alpha + betaS. The parameters alpha and beta are estimated using weighted linear regression with bootstrapping to get the standard errors, or using maximum likelihood. The authors show how the procedure works with continuous test data and with categorical test data.

Alcoholism↗

PubMedDate not supplied

Analyzing a portion of the ROC curve.

The area under the ROC curve is a common index summarizing the information contained in the curve. When comparing two ROC curves, though, problems arise when interest does not lie in the entire range of false-positive rates (and hence the entire area). Numerical integration is suggested for evaluating the area under a portion of the ROC curve. Variance estimates are derived. The method is applicable for either continuous or rating scale binormal data, from independent or dependent samples. An example is presented which looks at rating scale data of computed tomographic scans of the head with and without concomitant use of clinical history. The areas under the two ROC curves over an a priori range of false-positive rates are examined, as well as the areas under the two curves at a specific point.

Computer Simulation↗

PubMedDate not supplied

Integrating stratum-specific likelihood ratios with the analysis of ROC curves.

Data used to construct receiver operating characteristic (ROC) curves and to calculate the area under the curve (ROC AUC) can be used to derive stratum-specific likelihood ratios (SSLRs) with their 95% confidence intervals (95% CIs). The purpose of this study was to determine whether useful information can be obtained by adding SSLRs to the analysis of ROC curves. The authors analyzed four previously reported sets of data: 1) serum creatine kinase (SCK) for diagnosing acute myocardial infarction (AMI) in the coronary care unit (CCU); 2) SCK in the evaluation of chest pain in the emergency center (EC); 3) four predictor variables in the diagnosis of strep throat; and 4) the ordinal assessment of computed tomographic (CT) images. Use of SCK in the CCU produced four strata that had posttest probabilities that were highly discriminating, whereas SCK in the EC resulted in only two strata with limited discriminating ability. In either study the cutpoint at which the SSLR changed from less than to greater than 1.0 was higher than the reported upper normal for the test, thereby quantitating spectrum bias. The maximum number of strata of predictor signs and symptoms for strep throat was three rather than the five used in previous studies. With a larger sample size or pooling, four strata could probably be developed. With CT images, "definitely normal," "probably normal," and "questionable" were collapsed to one negative stratum. "Probably abnormal" became the true "questionable" stratum and "definitely abnormal" was the only positive stratum. The authors conclude that additional useful information is obtained by deriving stratum-specific likelihood ratios as part of the analysis of an ROC curve.

Humans↗

PubMedDate not supplied

Testing an underlying assumption on a ROC curve based on rating data.

The receiver operating characteristic (ROC) curve is widely accepted as the measurement that best represents the accuracy of a diagnostic system that consists of a diagnostic imaging modality and a reader who is interpreting results from the modality. However, there is one implicit assumption in a ROC curve: that the cutoff points to declare a positive test for decision variables of a diseased patient and a nondiseased patient are the same. If this assumption is violated, then a ROC curve does not exist. The author develops a statistical procedure to test this assumption. To demonstrate, he applies the method to a ROC data set and shows that the assumption of the equal cutoff points could be rejected.

Data Interpretation, Statistical↗

PubMedDate not supplied

Statistical comparison of ROC curves from multiple readers.

Receiver operating characteristic (ROC) analysis is the commonly accepted method for comparing diagnostic imaging systems. In general, ROC studies are designed in such a way that multiple readers read the same images and each image is presented by means of two different imaging systems. Statistical methods for the comparison of the ROC curves from one reader have been developed, but extension of these methods to multiple readers is not straightforward. A new method of analysis is presented for the comparison of ROC curves from multiple readers. This method includes a non-parametric estimation of the variances and covariances between the various areas under the curves. The method described is more appropriate than the paired t test, because it also takes the case-sample variation into account.

Breast Neoplasms↗

PubMed1996

Spatial resolution requirements in digital radiography of scaphoid fractures. An ROC analysis.

PURPOSE: To investigate the spatial resolution requirements in digital radiography of scaphoid fractures. MATERIAL AND METHODS: Included in the study were 60 scaphoid radiographs with and 60 without fractures of the scaphoid bone. The film-screen images were digitized using pixel sizes of 115, 170, and 340 microns along with 170 microns with a 10:1 wavelet compression. The digital images were displayed on a 1280 x 1024 x 8 bits monitor, and 5 observers evaluated the images in 5 randomized sessions. The results for each pixel size were then compared to the film-screen images by ROC analysis. RESULTS: The mean area under the ROC curves was larger for the film-screen images than for the digital images at all resolutions. However, this difference was not significant when the areas under the ROC curves for the film-screen images were compared to the digital images of 115, 170, and 170 microns with 10:1 compression. There was a significant difference for the 340-microns pixel size in favour of the film-screen images. The mean ROC curves for the digital images were very similar for the 115 and 170 microns pixel sizes, although slightly better for 115 microns. At 170 microns, the compression seemed to have a relatively small negative effect on the diagnostic performance; the deterioration was greater when the pixel size was increased to 340 microns. There was no obvious correlation between diagnostic performance and the experience of the observers in using work-stations. CONCLUSIONS: The pixel size of 170 microns is adequate for the detection of subtle fractures, even after wavelet compression by a ratio of 10:1.

Angiography, Digital Subtraction↗

PubMed1994

ROC curve regression analysis: the use of ordinal regression models for diagnostic test assessment.

Diagnostic tests commonly are characterized by their true positive (sensitivity) and true negative (specificity) classification rates, which rely on a single decision threshold to classify a test result as positive. A more complete description of test accuracy is given by the receiver operating characteristic (ROC) curve, a graph of the false positive and true positive rates obtained as the decision threshold is varied. A generalized regression methodology, which uses a class of ordinal regression models to estimate smoothed ROC curves has been described. Data from a multi-institutional study comparing the accuracy of magnetic resonance (MR) imaging with computed tomography (CT) in detecting liver metastases, which are ideally suited for ROC regression analysis, are described. The general regression model is introduced and an estimate for the area under the ROC curve and its standard error using parameters of the ordinal regression model is given. An analysis of the liver data that highlights the utility of the methodology in parsimoniously adjusting comparisons for covariates is presented.

Algorithms↗

PubMed1994

The ROC curve in the evaluation of fasting capillary blood glucose as a screening test for diabetes and IGT.

OBJECTIVE: To assess the performance of different cutoff points of fasting capillary glycemia (FCG) in the diagnosis of diabetes and impaired glucose tolerance (IGT) using the receiver operating characteristics (ROC) curve approach. RESEARCH DESIGN AND METHODS: This study included a sample of 4,019 subjects without a previous history of diabetes who were recruited for a confirmatory 2-h post-glucose challenge capillary glycemia from a larger sample of 21,847 individuals screened with FCG. The sensitivity and the specificity of FCG as a screening test were analyzed in the cutoff range 3.9-8.9 mmol/l, and the corresponding ROC curves were plotted to assess the performance of the test. RESULTS: The screening test performance was better for diabetes than for IGT in the full range of cutoff points studied. Sensitivities ranged between 37.9 and 97.1% for diabetes and 1.8 and 94.4% for IGT; the areas under the ROC curves were 0.91 +/- 0.01 and 0.75 +/- 0.01, respectively. The cutoffs showing the best equilibrium between sensitivity and specificity approached 5.6 and 5.0 mmol/l for diabetes and IGT, respectively. Factors such as age, color, and family history of diabetes can affect the screening test performance. CONCLUSIONS: ROC curves can provide useful information toward improving the usefulness of FCG as a screening test for abnormalities of glucose tolerance.

Adult↗

PubMed1993

Diagnostic accuracy evaluation using ROC curve analysis.

The diagnostic accuracy of a biochemical quantity is inversely related to the overlapping zone between the values of the population suffering from a disease and the population which does not. The ROC curves are an indirect measure of the overlapping zone between both populations. Specimens (plasma and urine) taken from 928 patients with symptoms of acute abdominal pain were used and the catalytic concentration of alpha-amylase, pancreatic alpha-amylase and triacylglycerol lipase (determined by two methods) were measured. Definitive diagnosis was obtained by following the directives of expert groups on the evaluation of diagnostic tests. Diagnostic accuracy was characterized by calculating the diagnostic sensitivity and specificity, by representing the ROC curves and by quantifying the areas under the ROC curves. The catalytic concentration of pancreatic alpha-amylase in plasma was the quantity with a greater area under the ROC curve (A = 0.9740) and then the one which had greatest diagnostic accuracy. If we considered the upper limit of the reference interval to be the cut-off value, the catalytic concentration of pancreatic alpha-amylase in plasma had a diagnostic sensitivity and specificity values of 0.96 and 0.88 respectively for the acute pancreatitis.

Acute Disease↗

PubMed1990

On the shape of energy-detection ROC curves.

A numerical study shows that energy-detection ROC curves based on chi-squared distributions may be accurately represented by binormal receiver operating characteristic (ROC) curves. This allows the detector accuracy and the ROC shape (asymmetry) to be expressed simply in terms of the distribution parameters. An established approximate formula for the shape of energy-detection ROC curves is shown to be inaccurate. An improved approximately is presented.

Attention↗

PubMed1997

Recognition memory ROCs for item and associative information: the contribution of recollection and familiarity.

Receiver-operating characteristics (ROCs) were examined in three recognition memory experiments. ROCs for item information (i.e., was this word presented?) were found to be curvilinear. However, ROCs for associative information (i.e., were these two words presented together?) were found to be linear. The results are in agreement with the predictions of a dual-process model that assumes that recognition judgments are based on familiarity and recollection. Familiarity reflects the assessment of a continuous strength dimension and is well described as a signal detection process, whereas recollection reflects the retrieval of qualitative information about the study episode and behaves like a discrete threshold process. The results showed that memory judgments about items relied on a combination of recollection and familiarity, but that judgments about associations relied primarily on recollection. Further examination of the associative ROCs suggested that subjects were able to recollect that old pairs of items were in the study list, and, under some conditions, that new pairs were not in the study list.

Adult↗

PubMed1999

[Generalized ROC criteria in the evaluation of several tumor markers].

The main objective of this paper is to present a method of evaluating several tumor markers by using the generalized ROC criterion. This criterion finds the best linear combination of the tumor markers such that the area under the ROC curve is maximized. Confidence intervals for the generalized ROC criteria are also presented. This methodology is applied to 51 patients with advanced colorectal cancer for whom the ACE tumor markers were measured before and during chemotherapy treatment. Two populations were defined according to clinical response to chemotherapy. Each marker taken separately, whether on the raw scale or on the transformed scale, contained 0.5 in the confidence interval and was thus non significant. This was also true for both markers on the raw scale. However, the best linear combination on the logarithms of ACE before and at evaluation gave a significantly better area under the ROC curve. A weighted change in ACE measurements significantly distinguishes between responders and non responders in patients with advanced colorectal cancer. We propose that the methodology presented in this paper be used for the evaluation of several tumor markers.

Analysis of Variance↗

PubMed2000

[ROC analysis comparing screen film mammography and digital mammography].

PURPOSE: To compare the diagnosis performances of radiologists on screen film versus digital mammography. MATERIAL: and methods: Two sets of 123 mammograms, screen film mammography and storage phosphor digital mammography, are studied comparatively with ROC analysis. RESULTS: Phantom study show that conventional method give better scores for usual tension but the detectability of smaller microcalcification is equivalent. To obtain with digital technic the same conventional score you have to increase the radiation dose. Roc Curves, simulated "detection" mode showed that radiologists performed with higher accuracy with conventional system but this difference is weekly statistically significant. ROC Curves, simulated "diagnostic" mode showed the same results wit no statistically significant difference but when the decision to go to the biopsy is the gold standard, ROC Curves were essentially equivalent for both film screen and digital mammography system. The readers consistently considered the digital mammograms to be less suspicious for cancer findings. The agreement study as proposed by the FDA indicate that probability of a positive digital mammograms given a positive screen film is 75% (threshold value 90%) and the probability of a negative digital mammograms given a negative analog film is 85% (threshold value 85%). CONCLUSION: Analysis of specific discrepancies indicate that spatial resolution is an essential limiting factor for digital method but high resolution phosphor plate are interesting in imaging treated breast, radioluscent lesion, fatty benign tumor, hamartoma, intramammary lymph node, breast with prosthesis.

Biopsy↗

PubMed1990

[ROC-curve analysis. A statistical method for the evaluation of diagnostic tests].

Receiver operating characteristic (ROC)-curves are a statistical method which may be employed inter alii for assessing a diagnostic test. These have been employed particularly in radiology but have also been employed for assessing laboratory tests and obstetric estiamation scales. The method is based on an analysis of nosological probabilities for varying limits of decision thresholds. ROC-curves may be employed for data on ratio/interval scales and data on rank scales. Parametric and non-parametric methods are available for obtaining a single quantitative measurement for the entire ROC-curve and to carry out significance tests between several ROC-curves.

Diagnosis↗

PubMed1989

Receiver operating characteristic (ROC) methodology: the state of the art.

Receiver operating characteristic (ROC) methodology has been increasingly used in medical applications in the last 10 years. The text by Swets and Pickett has popularized the technique and the journal Medical Decision Making (1981--) provides a forum for further methodologic issues. In this article, I will (1) describe the nature of the data generated by ROC studies; (2) evaluate the choices of summary indices of performance (accuracy); (3) outline the data-analytic techniques used, and how to incorporate data from multiple observers and multiple "readings"; (4) review proposed alternatives to the commonly used binormal ROC model; and (5) discuss issues, such as verification bias, and challenges, such as multicenter comparative imaging studies and the difficulty of obtaining "truth data", which need to be addressed when adapting ROC methods to medical contexts.

ROC Curve↗

PubMed1989

[On the assessment of the diagnostic accuracy of imaging diagnosis by ROC and BVC analyses--in reference to X-ray CT and ultrasound examination of liver disease].

To achieve an effective assessment of diagnostic accuracies of currently used imaging methods on liver diseases, X-ray CT (CT) and realtime ultrasound (US), ROC analysis and a new analysis method, "the bias to variance characteristics (BVC)" were applied in parallel. The survey has been carried out to assess an image interpretation accuracy on cases with established diagnosis collected on the multi-institutional basis. The images, CT and US of liver diseases including localized and diffuse lesions were selected for image reading sessions and the interpretative difficulties were geared to an appropriate grade so that it does fit to the basic requirement of ROC analysis. Consequently, 91 cases were selected from a total of 258 cases which were collected from 16 institutions. Two image reading sessions were scheduled and a total of 28 radiologists and sonologists undertook the first session. The final analysis covered 8 radiologists for CT and 9 sonologists who carried out both sessions. The image interpretation results were fed to the computer and analyses were made accordingly. Analysis of the first image reading session gave true positive ratios (TPR) of CT and US as 92.5% and 97.4%, and true negative ratios (TNR) of 50.5% and 46.8%, respectively. The second reading session, in which both CT and US images were interpreted simultaneously, gave the similar values of TPRs, 96-97% and TNRs, 44-47%. The ROC analysis of existence of liver mass and diagnosis of malignancy resulted in the higher values in US compared to CT in the first separate reading, however, the 2nd session gave almost identical values in each modality. The diagnostic accuracy for the benign tumors, mostly consisted of hemangioma, CT reading gave higher accuracy rates compared to the US. The entirely new interpretations scheme, arbitrarily named as BVC was carried ou on matrix data obtained in the same experiment. The scatterograms plotted according to the BV values, the mean and the standard deviation inherent to individual cases have demonstrated marked variations among individual cases most likely reflecting the interpretative difficulty of respective images, thus indicating the method to be well in adjunct to the ROC analysis.

Adult↗