SEARCH · PubMed Health

Results for “roc”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9Linked to original sources

PubMed1993

Receiver operating characteristic (ROC) curve analysis of the tumour markers CEA, CA 50 and CA 242 in pancreatic cancer; results from a prospective study.

The serum values of the tumour markers carcinoembryonic antigen (CEA), cancer-associated carboanhydrate antigens CA 50 and CA 242 were evaluated in 193 patients with hepatopancreato-biliary diseases by receiver operating characteristic (ROC) curve analysis in order to compare their diagnostic accuracy in pancreatic cancer (n = 26), and to define optimal cut-off levels for the serum values of these tumour markers in the diagnosis of pancreatic cancer. The ROC analysis showed that all marker tests are considerably sensitive (77-81%) at the specificity level of 80%. The CA 242 test was more sensitive than CEA and CA 50 at high specificity levels (> 0.90) but slightly less sensitive at low specificity levels (< 0.60). The CEA test and CA 50 test performed equally well at high and low specificity levels. According to this study, it would seem optimal to use the cut-off level of 4.1 ng ml-1 for CEA, and the level of 137 U ml-1 for CA 50, since they gave a sensitivity of 77% at the specificity levels of 83% and 84%, respectively. For CA 242 the optimal cut-off level was 21 U ml-1, which gave a sensitivity and specificity of 81%. In conclusion, the results of ROC curve analysis suggest that the CA 242 test has an advantage over CEA and CA 50 because of its higher specificity in pancreatic cancer. In addition, it would seem reasonable to use higher cut-off values than what has been recommended for CEA and CA 50 in the diagnosis of pancreatic cancer, but for CA 242 the recommended cut-off level of 20 U ml-1 seems appropriate.

Antigens, Tumor-Associated, Carbohydrate↗

PubMed2000

Diagnostic accuracy of helical CT arterial portography and CT hepatic arteriography for hypervascular hepatocellular carcinoma in chronic liver damage. An ROC analysis.

PURPOSE: To evaluate the detectability of hypervascular hepatocellular carcinomas (HCCs) in chronic liver damage with helical CT arterial portography (CTAP) and CT hepatic arteriography (CTHA). MATERIAL AND METHODS: Thirty-nine HCC patients who underwent CTAP and CTHA were studied. Diagnostic abilities of CTAP alone, CTHA alone, or combined CTAP and CTHA were evaluated by receiver operating characteristic (ROC) analysis. Fifty-three images with 53 HCC nodules were evaluated. Tumor size ranged from 5 to 90 mm (mean 22.8 mm). Sensitivities and specificities for all techniques were calculated. RESULTS: ROC analysis showed the diagnostic ability significantly better with combined CTAP and CTHA (mean area under the ROC curve (Az)=0.95), or CTHA alone (Az=0.93) than CTAP alone (Az=0.87) (p<0.01). Combined CTAP and CTHA showed the best sensitivity (95.0%), followed by CTHA alone (88.1%) and CTAP alone (85.5%). The specificities of all three imaging techniques were relatively low (54.1% for combined CTAP and CTHA, 71.1% for CTHA alone, and 54.1% for CTAP alone) because of perfusion abnormalities of the liver parenchyma. CONCLUSION: The combination of CTAP and CTHA is superior to CTAP alone for detection of hypervascular HCCs. However, its specificity was relatively low in chronic liver damage.

Aged↗

PubMed2004

Significance analysis of ROC indices for comparing diagnostic markers: applications to gene microarray data.

A common objective in microarray experiments is to select genes that are differentially expressed between two classes (two treatment groups). Selection of differentially expressed genes involves two steps. The first step is to calculate a discriminatory score that will rank the genes in order of evidence of differential expressions. The second step is to determine a cutoff for the ranked scores. Summary indices of the receiver operating characteristic (ROC) curve provide relative measures for a ranking of differential expressions. This article proposes using the hypothesis-testing approach to compute the raw p-values and/or adjusted p-values for three ROC discrimination measures. A cutoff p-value can be determined from the (ranked) p-values or the adjusted p-values to select differentially expressed genes. To quantify the degree of confidence in the selected top-ranked genes, the conditional false discovery rate (FDR) over the selected gene set and the "Type I" (false positive) error probability for each selected gene are estimated. The proposed approach is applied to a public colon tumor data set for illustration. The selected gene sets from three ROC summary indices and the commonly used two-sample t-statistic are applied to the sample classification to evaluate the predictability of the four discrimination measures.

Algorithms↗

PubMed1998

An evaluation of the role of ROC plots in the prediction of heroin use from total codeine and total morphine concentrations in urine.

A diagnostic system to predict the presence of 6-acetylmorphine (6AM) in opiate-positive urines was recently proposed. A twofold criterion based on the total morphine concentration and the total codeine to total morphine concentration ratio was identified. Using relative operating characteristic (ROC) analysis, it was determined that the diagnostic system had a sensitivity of 92%, a specificity of 79%, and an overall accuracy of 73%. We applied similar decision criteria to a study population of 125 opiate-positive urines collected from criminal justice clients of a West Coast reference laboratory. ROC analysis on this population produced very different results: a sensitivity of 91%, a specificity of 49%, and an accuracy of 45%. These data illustrate the importance of choosing a representative study population without any selection biases that may compromise the validity of the accuracy measure. The ROC plot is an important tool for assessing a clinical test's performance, but in order for toxicologists and Medical Review Officers to benefit from the diagnostic test results, they must also know the predictive value (based on test accuracy and the prevalence of heroin use) of the test results. They need to know how well the test predicts the presence of 6AM, and therefore, the illicit use of heroin, in the population of interest whether it be workplace, criminal justice, hospital emergency department clients, or a combination of all populations.

Cocaine-Related Disorders↗

PubMed1992

Screening test length for sequential testing with a standardized-patient examination: a receiver operating characteristic (ROC) analysis.

The feasibility of using sequential testing (i.e., using a screening test) to reduce the length and expense of a performance-based examination with standardized-patient cases was demonstrated previously. In the present study, quantitative criteria rather than practical considerations were used to determine optimal values for the length of the screening test (i.e., number of cases) and the location of the screening pass-fail cutoff (i.e., its relation to the mean of the pass levels for the different cases). Data were derived from five classes of senior students at the Southern Illinois University School of Medicine, 1987-1991. Specifically, receiver operating characteristic (ROC) curves were plotted for screening tests of varying lengths, with the points on each ROC curve corresponding to different pass-fail cutoffs on the screening test. The results showed that good accuracy can be attained with a screening test that is only one-third the length of the full examination and that the cutoff for this screen should be set slightly above the mean of the case pass levels to maximize sensitivity and specificity. The authors conclude that their study demonstrates the value of an ROC analysis in evaluating the psychometric properties of a screening test in sequential testing.

Clinical Clerkship↗

PubMed1990

Nonparametric estimation of degenerate ROC data sets used for comparison of imaging systems.

Maximum likelihood estimation, assuming a binormal model, has been used extensively to estimate the area under receiver operating characteristic (ROC) curves and thus provide an index of diagnostic accuracy for the comparison of two imaging systems. However, in some instances, a degenerate data set results in a fitted ROC curve of inappropriate shape. In those instances, a nonparametric Wilcoxon statistic may be used to compare the areas under two ROC curves. Simulation of this approach indicates that the procedure has relatively high statistical power and its use in specific degenerate data sets shows its applications potential.

Data Interpretation, Statistical↗

PubMed1995

The comparison of injury severity instrument performance using likelihood ratio and ROC curve analyses.

BACKGROUND: The relative merits of injury severity instruments are often difficult to determine because the statistical techniques, study populations, and outcomes used in trauma research tend to obscure rather than elucidate differences in performance. OBJECTIVE: To describe the advantages of likelihood ratio and receiver operator characteristic (ROC) curve analyses and to demonstrate them using study populations and outcomes that facilitate instrument discrimination. MATERIALS AND METHODS: Previously published data on the performance of the "Injury Severity Score" and of four triage instruments in the prediction of mortality; paired comparison of two instruments in a previously unpublished trauma registry dataset; use of likelihood ratio and ROC analyses. MEASUREMENTS AND MAIN RESULTS: In a comparison of triage instruments this study clearly showed that CRAMS, PHI, and RTI, which contain nonphysiological information, have higher performance levels than the Revised Trauma Score (RTS), which is restricted to physiological information. Absolute performance gains of these instruments over the RTS ranged from 5.9% to 43.5%. This study also noted that the ISS performs equally well for blunt and penetrating trauma, and affirms concerns about the adequacy of its performance. CONCLUSIONS: Likelihood ratio and ROC curve analyses demonstrate differences in injury severity instrument performance that were obscured by less rigorous methods of comparison.

Humans↗

PubMed2003

ROC analysis for assessment of lesion detection performance in 3D PET: influence of reconstruction algorithms.

Image quality in positron emission tomography (PET) can be assessed with physical parameters, as spatial resolution and signal-to-noise ratio, or using psychophysical approaches, which include the observer performance and the considered task (ROC analysis). For PET in oncology, such a task is the detection of hot lesions. The aim of the present study was to assess the lesion detection performance due to adequate modeling of the scanner and the measurement process in the image reconstruction process. We compared the standard OSEM software of the manufacturer with a sophisticated fully 3D iterative reconstruction technique (USC MAP). A rectangular phantom with 6 oblique line sources in a homogeneous background (2.6 kBq/ml 18F) was imaged dynamically with an ECAT EXACT HR+ scanner in 3D mode. Reconstructed activity contrasts varied between 15 and 0, as the line sources were filled with 11C (3.2 MBq/ml). Measured attenuation and standard randoms, dead time, and scatter corrections of the manufacturer were employed. For the ROC analysis, a software tool presented a cut-out of the phantom (15 x 15 pixels) to two observers. These cut-outs were rated (5 classes) and the area Az under the ROC curve was determined as a measure of detection performance. The improvement for Az with USC MAP compared to the OSEM reconstructions ranged between 0.02 and 0.23 for signal-to-noise ratios of the background between 2.8 and 3.1 and lesion contrast between 2.1 and 4.2. This study demonstrates that adequate modeling of the measurement process in the reconstruction algorithm improves the detection of small hot lesions markedly.

Algorithms↗

PubMed2005

Computer-aided detection of peripheral lung cancers missed at CT: ROC analyses without and with localization.

PURPOSE: To retrospectively evaluate whether a difference-image computer-aided detection (CAD) scheme can help radiologists detect peripheral lung cancers missed at low-dose computed tomography (CT). MATERIALS AND METHODS: Institutional review board approval and informed patient and observer consent were obtained. Seventeen patients (eight men and nine women; mean age, 60 years) with a missed peripheral lung cancer and 10 control subjects (five men and five women; mean age, 63 years) without cancer at low-dose CT were included in an observer study. Fourteen radiologists were divided into two groups on the basis of different image display formats: Six radiologists (group 1) reviewed CT scans with a multiformat display, and eight radiologists (group 2) reviewed images with a "stacked" cine-mode display. The radiologists, first without and then with the CAD scheme, indicated their confidence level regarding the presence (or absence) of cancer and the most likely position of a lesion on each CT scan. Receiver operating characteristic (ROC) curves were calculated without and with localization to evaluate the observers' performance. RESULTS: With the CAD scheme, the average area under the ROC curve improved from 0.763 to 0.854 for all radiologists (P = .002), from 0.757 to 0.862 for group 1 (P = .04), and from 0.768 to 0.848 for group 2 (P = .01). The average sensitivity in the detection of 17 cancers improved from 52% (124 of 238 observations) to 68% (163 of 238 observations) for all radiologists (P < .001), from 49% (50 of 102 observations) to 71% (72 of 102 observations) for group 1 (P = .02), and from 54% (74 of 136 observations) to 67% (91 of 136 observations) for group 2 (P = .006). The localization ROC curve also improved. CONCLUSION: Lung cancers missed at low-dose CT were very difficult to detect, even in an observer study. The use of CAD, however, can improve radiologists' performance in the detection of these subtle cancers.

Aged↗

PubMed1999

Improvement of radiologists' characterization of mammographic masses by using computer-aided diagnosis: an ROC study.

PURPOSE: To evaluate the effects of computer-aided diagnosis (CAD) on radiologists' classification of malignant and benign masses seen on mammograms. MATERIALS AND METHODS: The authors previously developed an automated computer program for estimation of the relative malignancy rating of masses. In the present study, the authors conducted observer performance experiments with receiver operating characteristic (ROC) methodology to evaluate the effects of computer estimates on radiologists' confidence ratings. Six radiologists assessed biopsy-proved masses with and without CAD. Two experiments, one with a single view and the other with two views, were conducted. The classification accuracy was quantified by using the area under the ROC curve, Az. RESULTS: For the reading of 238 images, the Az value for the computer classifier was 0.92. The radiologists' Az values ranged from 0.79 to 0.92 without CAD and improved to 0.87-0.96 with CAD. For the reading of a subset of 76 paired views, the radiologists' Az values ranged from 0.88 to 0.95 without CAD and improved to 0.93-0.97 with CAD. Improvements in the reading of the two sets of images were statistically significant (P = .022 and .007, respectively). An improved positive predictive value as a function of the false-negative fraction was predicted from the improved ROC curves. CONCLUSION: CAD may be useful for assisting radiologists in classification of masses and thereby potentially help reduce unnecessary biopsies.

Breast↗

PubMed2006

Receiver operating characteristic (ROC) analysis for characterizing synaptic efficacy.

The role of background synaptic activity in cortical processing has recently received much attention. How do individual neurons extract information when embedded in a noisy background? When examining the impact of a synaptic input on postsynaptic firing, it is important to distinguish a change in overall firing probability from a true change in neuronal sensitivity to a particular input (synaptic efficacy) that corresponds to a change in detection performance. Here we study the impact of background synaptic input on neuronal sensitivity to individual synaptic inputs using receiver operating characteristic (ROC) analysis. We use the area under the ROC curve as a measure of synaptic efficacy, here defined as the ability of a postsynaptic action potential to identify a particular synaptic input event. An advantage of using ROC analysis to measure synaptic efficacy is that it provides a measure that is independent of postsynaptic firing rate. Furthermore, changes in mean excitation or inhibition, although affecting overall firing probability, do not modulate synaptic efficacy when measured in this way. Changes in overall conductance also affect firing probability but not this form of synaptic efficacy. Input noise, here defined as the variance of the input current, does modulate synaptic efficacy, however. This effect persists when the change in input variance is coupled with a change in conductance (as would result from changing background activity).

Algorithms↗

PubMed1991

ROC curves, test accuracy, and the description of diagnostic tests.

Clinicians can gain an enhanced understanding of the role of diagnostic tests once they are familiar and comfortable with the descriptions of test performance provided by receiver operating characteristic (ROC) analysis. This article explores the ways that ROC methods quantify test accuracy and describes how ROC methods characterize the distributions of test outcomes in study populations.

Humans↗

PubMedDate not supplied

Determining a range of false-positive rates for which ROC curves differ.

Many indices have been proposed for summarizing the information contained in the ROC curve. When comparing two ROC curves, though, there are times when global summary measures are either not optimal or not appropriate. The author presents a method for directly comparing true-positive rates for two diagnostic, screening or prognostic tools, determining over what range of false-positive values the tests differ. The method is applicable for independent or dependent samples. An example concerning gallium citrate imaging is presented, as well as an example using a prognostic index for severity of illness in the ICU. The range of false-positive rates for which the ROC curves differ is determined for each example.

Bacterial Infections↗

PubMedDate not supplied

The area under the ROC curve and its competitors.

The area under the receiver operating characteristic (ROC) curve is a popular measure of the power of a (two-disease) diagnostic test, but it is shown here to be an inconsistent criterion: tests of indistinguishable clinical impacts may have different areas. A class of diagnosticity measures (DMs) of proven optimality is proposed instead. Once a regret(-like) measure of diagnostic uncertainty is agreed upon, the associated DM is uniquely defined and, indeed, calculable from the ROC curve configuration. Two scaled variants of the ROC are introduced and used to advantage in the analysis. They may also be helpful to students of medical decision making.

Decision Theory↗

PubMedDate not supplied

Evaluation of medical technologies: a generalized ROC analysis.

The author describes a new methodology to solve medical decision problems involving a choice between alternatives under conditions of risk and uncertainty when imperfect empirical information from diagnostic technologies is available. The main new concept is the generalized ROC (GROC) curve, which extends considerably the scope of analysis of the traditional ROC curve as well as the threshold approach to medical decision making. Both techniques become special cases of the new general approach. The author shows how to apply the technique and derive comprehensive clinical guidelines to solve the medical decision problem involving one patient in the most general situation. The larger problem of evaluating the performances of diagnostic technologies for a population of patients with varying prior probabilities of disease is then addressed. A new measure of performance is proposed that goes beyond the well-accepted area under the ROC curve index. This measure is used to compare different technologies for a population of patients, and a new methodology is proposed to carry out such comparisons.

Cost-Benefit Analysis↗

PubMedDate not supplied

Empirical Bayes combination of estimated areas under ROC curves using estimating equations.

A synthesis of the empirical Bayes method and the method of estimating equations is used to combine individual receiver operating characteristic (ROC) area estimates from different studies of the same diagnostic test into a single estimate. This single estimate represents the population mean from which individual areas under the ROC curves were sampled. The only data needed to carry out the method are estimated areas under the ROC curves and the corresponding standard errors.

Bayes Theorem↗

PubMedDate not supplied

Statistical comparison of two ROC-curve estimates obtained from partially-paired datasets.

The authors propose a new generalized method for ROC-curve fitting and statistical testing that allows researchers to utilize all of the data collected in an experimental comparison of two diagnostic modalities, even if some patients have not been studied with both modalities. Their new algorithm, ROCKIT, subsumes previous algorithms as special cases. It conducts all analyses available from previous ROC software and provides 95% confidence intervals for all estimates. ROCKIT was tested on more than half a million computer-simulated datasets of various sizes and configurations representing a range of population ROC curves. The algorithm successfully converged for more than 99.8% of all datasets studied. The type I error rates of the new algorithm's statistical test for differences in Az estimates were excellent for datasets typically encountered in practice, but diverged from alpha for datasets arising from some extreme situations.

Algorithms↗

PubMedDate not supplied

A Bayesian approach to a general regression model for ROC curves.

A fully Bayesian approach to a general nonlinear ordinal regression model for ROC-curve analysis is presented. Samples from the marginal posterior distributions of the model parameters are obtained by a Markov-chain Monte Carlo (MCMC) technique--Gibbs sampling. These samples facilitate the calculation of point estimates and credible regions as well as inferences for the associated areas under the ROC curves. The analysis of an example using freely available software shows that the use of noninformative vague prior distributions for all model parameters yields posterior summary statistics very similar to the conventional maximum-likelihood estimates. Clinically important advantages of this Bayesian approach are: the possible inclusion of prior knowledge and beliefs into the ROC analysis (via the prior distributions), the possible calculation of the posterior predictive distribution of a future patient outcome, and the potential to address questions such as: "What is the probability that a certain diagnostic test is better in one setting than in another?"

Bayes Theorem↗