SEARCH · PubMed Health

Results for “roc”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8Linked to original sources

PubMed2004

Advantages to transforming the receiver operating characteristic (ROC) curve into likelihood ratio co-ordinates.

Traditionally, the receiver operating characteristic (ROC) curve for a diagnostic test plots true positives (sensitivity) against false positives (one minus specificity). However, this representation brings with it several drawbacks. A transformation to positive and negative likelihood ratio co-ordinates, scaled by base-ten logarithms, offers several advantages. First we motivate the use of positive and negative likelihood ratios, emphasizing their relationship to modification of the odds ratio. Then we highlight properties of likelihood ratios using the traditional ROC axes. Finally, we demonstrate ROC curves and their properties after conversion to likelihood ratio co-ordinates. These graphs do not waste space for tests lacking diagnostic power, and offer a simple visual assessment of a test's impact on the odds ratio.

Diagnostic Tests, Routine↗

PubMed2006

A Bayesian hierarchical approach to multirater correlated ROC analysis.

In a common ROC study design, several readers are asked to rate diagnostics of the same cases processed under different modalities. We describe a Bayesian hierarchical model that facilitates the analysis of this study design by explicitly modelling the three sources of variation inherent to it. In so doing, we achieve substantial reductions in the posterior uncertainty associated with estimates of the differences in areas under the estimated ROC curves and corresponding reductions in the mean squared error (MSE) of these estimates. Based on simulation studies, both the widths of coverage intervals and MSE of estimates of differences in the area under the curves appear to be reduced by a factor that often exceeds five. Thus, our methodology has important implications for increasing the power of analyses based on ROC data collected from an available study population.

Bayes Theorem↗

PubMed2006

Tests of equivalence and non-inferiority for diagnostic accuracy based on the paired areas under ROC curves.

Assessment of equivalence or non-inferiority in accuracy between two diagnostic procedures often involves comparisons of paired areas under the receiver operating characteristic (ROC) curves. With some pre-specified clinically meaningful limits, the current approach to evaluating equivalence is to perform the two one-sided tests (TOST) based on the difference in paired areas under ROC curves estimated by the non-parametric method. We propose to use the standardized difference for assessing equivalence or non-inferiority in diagnostic accuracy based on paired areas under ROC curves between two diagnostic procedures. The bootstrap technique is also suggested for both non-parametric method and the standardized difference approach. A simulation study was conducted empirically to investigate the size and power of the four methods for various combinations of distributions, data types, sample sizes, and different correlations. Simulation results demonstrate that the bootstrap procedure of the standardized difference approach not only can adequately control the type I error rate at the nominal level but also provides equivalent power under both symmetrical and skewed distributions. A numerical example using published data illustrates the proposed methods.

Computer Simulation↗

PubMed1994

Efficient confidence bounds for ROC curves.

The specificity and sensitivity of a quantitative diagnostic marker depends on the selected cut-off point. The ROC curve is generated by plotting sensitivity against specificity as the cut-off point runs through the whole range of possible marker values. Confidence bounds for ROC curves may be constructed by combining two separate confidence intervals for sensitivity and specificity. In the present paper, a more efficient method, which is based on a statistical test introduced by Greenhouse and Mantel, is presented. The resulting 95 per cent confidence intervals are up to 40 per cent smaller. A formula is given to calculate the sample size necessary to obtain a confidence interval of a stipulated length at a specified point of the ROC curve. As an example, the performance of a tumour marker in the diagnosis of bone marrow metastases is investigated.

Bone Marrow↗

PubMed1992

Receiver operating characteristic (ROC) analysis of the ability of arterial ketone body ratio to predict graft outcome after liver transplantation--its sensitivity and specificity.

To evaluate the ability of arterial ketone body ratio (AKBR; acetoacetate/3-hydroxybutyrate) to predict graft prognosis after liver transplantation, the diagnostic value as a predictive index was compared between AKBR and conventional liver function tests using receiver operating characteristic (ROC) analysis. The ROC curves were determined for AKBR, GOT, GPT, total bilirubin, serum lactate level, and prothrombin time, all of which were measured on the 1st and 2nd postoperative days in 88 cases of liver transplantation. Comparisons of the areas under the ROC curves between AKBR and other tests revealed the significant superiority of AKBR to other tests in predicting graft death within 1 month after transplantation. The present study suggests that AKBR can be used as an accurate index to predict graft prognosis after liver transplantation.

Adolescent↗

PubMed2005

ROC analysis as an additional method to characterize time to event data.

The receiver operating characteristic (ROC) is predominantly used to assess the discriminant power of diagnostic tests. The present paper proposes this method as an additional alternative to compare survival data of in oncology or related fields. Survival data of brain tumors were analysed with conventional Kaplan-Meier method and ROC. The Area und the Curve (ROC-curve) gives additional illustrative information to distinguish between two therapeutic approaches and low or high grade brain tumors.

Adult↗

PubMed2000

Discriminatory power of MRI for differentiation of adrenal non-adenomas vs adenomas evaluated by means of ROC analysis: can biopsy be obviated?

The purpose of our study was to evaluate the discriminatory power of MRI in high-field magnet (1.5 T) for differentiation of adrenal non-adenomas vs adenomas assessing the following parameters separately and in combination: mean diameter of adrenal mass; previously described and new ratios as well as index calculated from signal intensity (SI) on SE T2-weighted images, chemical shift imaging (CSI), and Gd-DTPA-enhanced dynamic studies. One hundred eight adrenal masses (36 non-hyperfunctioning adenomas, 27 pheochromocytomas, 23 aldosterone-secreting adenomas, 20 malignant masses and 2 cortisol-secreting adenomas) in 95 patients were evaluated with SE sequences, CSI and Gd-DTPA dynamic studies. Indices and ratios of SI for all examined MRI methods were calculated and examined retrospectively for significance of differences between the groups with calculation of sensitivity and specificity. Receiver operating characteristics (ROC) analysis of calculated parameters in combination was performed. The multifactorial analysis of all four parameters, including size of the tumor, T2(liver) index, CSI ratio reflecting lipid content in the tumor and Wo(max/last) ratio reflecting maximal washout of contrast agent from the tumor had 100 % sensitivity and 100 % specificity in characterization of adrenal non-adenoma. The best performance of combination of mean tumor diameter with single MRI SI parameter was achieved in combination with T2(liver) index for all adrenal masses (area under ROC 0.987) and CSI ratio for non-hyperfunctioning adrenal masses (area under ROC 0.991). Magnetic resonance imaging enables sensitive and specific diagnosis of adrenal non-adenoma.

Adenoma↗

PubMed2002

Computed tomography diagnosis utilizing compressed image data: an ROC analysis using acute appendicitis as a model.

Using receiver-operating characteristic (ROC) methodology, the ability to diagnose acute appendicitis with computed tomography (CT) images displayed at varying levels of lossy compression was evaluated. Nine sequential images over the ileocecal region were obtained from 53 consecutive patients with right lower quadrant pain who were clinically suspected to have acute appendicitis. Thirty were proven surgically to have acute appendicitis, alternative diagnoses confirmed in 23. The image sets were subjected to a lossy wavelet-based compression algorithm "Embedded Predictive Wavelet Image Coder" (EPWIC). Compression levels were: none, 8:1, 16:1, and 24:1, resulting in 4 sets of images per patient. Image sets were randomized and evaluated separately by 4 body radiologists on a 1,024 x 768-pixel SVGA color PC monitor in 512 x 512 format. The readers were aware of the clinical suspicion of appendicitis but were unaware of the positive fraction of cases. Individual and combined reader ROC and c2 analyses of sensitivity, specificity, and accuracy were determined. For all readers, sensitivity decreases at 16:1 and 24:1 levels (P <0.01, P <0.001, respectively). Accuracy decreased at 24:1 levels (P <0.01). Specificity was unaffected. By ROC analysis there was statistically significantly decreased area under the curve at 24:1 levels (P <0.02) as compared with uncompressed images. Finite levels of lossy wavelet compression may be applied to CT images without compromising diagnostic performance.

Acute Disease↗

PubMed1994

An Excel program for calculating and plotting receiver-operator characteristic (ROC) curves, histograms and descriptive statistics.

Receiver-operator characteristic (ROC) curves have been used increasingly to assess the performance of clinical laboratory tests and to determine suitable positive/negative threshold values. The Microsoft Excel 4.0 program Plot.ROC was designed to provide formatted, publication quality output with minimal user involvement. It asks for a few parameters at the beginning and then runs autonomously, creating ROC charts, histograms and descriptive statistics for each test chosen, and a cumulative ROC chart to compare tests. Output can easily be formatted further using Excel's graphics and text formatting capabilities.

Biomarkers, Tumor↗

PubMed1994

Evaluation of screening methods for Down's syndrome using bootstrap comparison of ROC curves.

This paper concerns the prediction of fetal Down's syndrome in pregnant women. Down's syndrome is the most common congenital cause of severe mental retardation. We elaborate two predictive functions of trisomy 21, combining maternal age and a maternal serum marker. We evaluated them by means of receiver operating characteristic (ROC) curves which give a representation of sensitivity and specificity of a prediction model when varying the cutoff of the predictor on the whole spectrum. Since normal statistical methods for comparison of ROC curves rely on distributional assumptions which were not verified, we used bootstrapping of ROC curves as a check for the statistical significance of differences between the areas under the curves.

Algorithms↗

PubMed1996

ROC analysis of noninvasive tests for peripheral arterial disease.

The purpose of this study is to evaluate the diagnostic accuracy of selected noninvasive tests for assessing peripheral arterial disease. The ankle/brachial index (ABI) and the femoral and popliteal pulsatility indices (PI), and combinations of these tests, were evaluated using receiver operating characteristic (ROC) analysis to determine their diagnostic accuracy depending on the localization of the disease. Verification bias, introduced by the preferential selection of patients for angiography based on the noninvasive test results, was examined. This study suggests that, in assessing whether a patient has significant peripheral arterial disease (lesions > or = 50%), determining an ABI is justified (ROC area 0.95 +/- 0.02). For disease localized to the aortoiliac segment, performing a single test, the femoral PI, is sufficient (ROC area 0.80 +/- 0.04). For disease including the femoropopliteal and infrapopliteal segments, a combination of tests is necessary. Utilized threshold values need to be adjusted for verification bias.

Adult↗

PubMedDate not supplied

Effects of optimization and image processing in digital chest radiography: an ROC study with an anthropomorphic phantom.

A digital system for chest radiography based on a large image intensifier was compared to a conventional film-screen system. The digital system was optimized with regard to spatial and contrast resolution and dose. The images were digitally processed for contrast and edge enhancement. A simulated pneumothorax and two simulated nodules were positioned over the lungs and the mediastinum of an anthropomorphic phantom. Observer performance was evaluated with ROC analysis. Five observers assessed the processed digital images and the conventional full-size radiographs. The time spent viewing the full-size radiographs and the digital images was recorded. For the simulated pneumothorax, the results showed perfect performance for the full-size radiographs and detectability was high also for the processed digital images. No significant difference in the detectability of the simulated nodules was seen between the two imaging systems. The results for the digital images showed a significantly improved detectability for the nodules in the mediastinum as compared to a previous ROC study where no optimization and image processing was available. No significant difference in detectability was seen between the former and the present ROC study for small nodules in the lung. No difference was seen in the time spent assessing the conventional full-size radiographs and the digital images. The study indicates that processed digital images produced by a large image intensifier are equal in image quality to conventional full-size radiographs for low-contrast objects such as nodules.

Evaluation Studies as Topic↗

PubMed1992

Cephalometrics of anterior open bite: a receiver operating characteristic (ROC) analysis.

A new approach is presented for the evaluation of cephalometric measurements in which a measurement is considered to be a diagnostic test for the presence or the absence of some component of malocclusion. This approach allows cephalometric measurements to be judged by the criteria that are generally used for clinical diagnostic tests, including determination of sensitivities, specificities, positive and negative predictive values, and, most important, receiver operating characteristic (ROC) curves. In this study, ROC curves are generated for the relationship between several skeletal cephalometric measurements and anterior dental open bite in a sample of 1541 orthodontic patients. The overbite depth indicator is found to be a better diagnostic criterion for the presence of dental open bite than any other commonly used skeletal cephalometric measurement or ratio. ROC curves are of substantial value for evaluating the diagnostic information of cephalometric measurements.

Adolescent↗

PubMed2005

The ROC analysis for different time points during oral glucose tolerance test.

This purpose was to discuss optimal reference value for normal glucose tolerance (NGT) at different time points during an oral glucose tolerance test (OGTT) by receiver operating characteristic (ROC) analysis. One thousand seven hundred and forty-six subjects from March 2004 to March 2005 were given 75 g OGTT and the blood samples were collected at 0, 30, 60, 120 and 180 min for glucose measurement. Other demographic data (e.g. age, sex, body mass index, BMI) were also recorded. The status of NGT, impaired glucose tolerance (IGT), impaired fasting glucose (IFG) and DM was determined according to American Diabetes Association (ADA) criteria. IGT and IFG were merged as impaired glucose regulation (IGR). Using IGR as the assumed test, the cutoff values at 0, 30, 60, 120 and 180 min during OGTT for normal reference range were calculated by ROC analysis. The cutoff values were 5.6, 9.5, 10.1, 7.8 and 6.1 mmol/l at 0, 30, 60, 120 and 180 min, respectively, in whole subjects. In addition, we provided the ROC information in age-stratified groups since there are differences among different age groups. When the results of OGTT were considered, the present study provided a reference range value to evaluate the status of glucose tolerance. Because of the heterogeneity of diabetes, further studies are necessity. And the sample collection is continuing.

Aged↗

PubMed2005

ROC curves and evaluation of radiation-induced pulmonary toxicity in breast cancer.

PURPOSE: To study clinical, radiologic, and physiologic pulmonary toxicity in 128 women after adjuvant radiotherapy (RT) for breast cancer in relation to dosimetric factors. METHODS AND MATERIAL: The patients underwent pulmonary function testing before and 5 months post-RT. Similarly, computer tomography of the chest was repeated 4 months post-RT and changes were scored with a semiquantitative system. Clinical symptoms were registered and scored according to Common Toxicity Criteria. All patients underwent three-dimensional dose planning, and the ipsilateral lung volume receiving > or = 13 Gy (V13), V20, and V30 were calculated. Multiple logistic or regression analyses were used for multivariate modeling. The relation between the dosimetric factors and side effects was also analyzed with receiver operating characteristic (ROC) curves. RESULTS: V20 was, according to multivariate modeling, the most important variable for the occurrence of the three studied side effects (p < 0.01). Age was also related to symptomatic and radiologic pneumonitis. Reduced pre-RT functional level was more common in patients developing symptomatic toxicity. The ROC areas for symptomatic pneumonitis in relation to V13, V20, and V30 were 0.69, 0.69, and 0.67, and for radiologic pneumonitis 0.85, 0.85, and 0.81. CONCLUSIONS: Our results support the use of three-dimensional planning aimed at minimizing the percent of incidentally irradiated lung volume to reduce pulmonary toxicity. Age was also correlated with post-RT side effects. According to ROC analysis, V20 could well predict the risk for radiologic pneumonitis for the studied semiquantitative model.

Age Factors↗

PubMed2001

ROC optimization may improve risk stratification of prostate cancer patients.

OBJECTIVES: Rational treatment decision requires accurate projection of the clinical course of a patient. Current methods in clinical outcome analysis mostly focus on population data. We investigated the applicability and optimization of the widely used actuarial method to project individual clinical outcomes. METHODS: We designed and implemented a Clinical Outcome Prediction Expert (COPE) that performs, assesses, and optimizes actuarial prediction on individual cases. We analyzed a post-prostatectomy database, consisting of 1043 patients. Sixty percent of the database was used for training and 40% for validation. Stratified actuarial curves are used to project individual outcomes. The prostate-specific antigen (PSA) level, the Gleason score, and the clinical American Joint Commission on Cancer Staging T-stage before treatment were used as predictors. The area under the receiver operator characteristic (ROC) curve was used to measure predictive performance. RESULTS: We obtained simple optimized stratification of pretreatment PSA level of 10 ng/mL or less, or more than 10 ng/mL; Gleason score of 6 or lower, or higher than 6; and clinical AJCC T-stage of T2a or lower, or higher. The optimized univariate risk scores were used to generate a multivariate score. After optimization, we found the higher risk group consisted of patients with PSA more than 10 ng/mL, or with PSA of 10 ng/mL or less and Gleason score higher than 6 and clinical AJCC T-stage higher than T2a. The optimized multivariate risk score has the highest ROC area of 0.77 among all predictors. CONCLUSIONS: The best conditions to perform actuarial prediction on individual cases are not known a priori and require optimization. This study shows that ROC optimization simplifies risk stratification and may improve the accuracy of clinical outcome prediction.

Data Interpretation, Statistical↗

PubMed1996

Use of receiver operating characteristic (ROC) analysis to evaluate sequence matching.

In this paper, we borrow the idea of the receiver operating characteristic (ROC) from clinical medicine and demonstrate its application to sequence comparison. The ROC includes elements of both sensitivity and specificity, and is a quantitative measure of the usefulness of a diagnostic. The ROC is used in this work to investigate the effects of scoring table and gap penalties on database searches. Studies on three families of proteins, 4Fe-4S ferredoxins, lysR bacterial regulatory proteins, and bacterial RNA polymerase sigma-factors lead to the following conclusions: sequence families are quite idiosyncratic, but the best PAM distance for database searches using the Smith-Waterman method is somewhat larger than predicted by theoretical methods, about 200 PAM. The length independent gap penalty (gap initiation penalty) is quite important, but shows a broad peak at values of about 20-24. The length dependent gap penalty (gap extension penalty) is almost irrelevant suggesting that successful database searches rely only to a limited degree on gapped alignments. Taken together, these observations lead to the conclusion that the optimal conditions for alignments and database searches are not, and should not be expected to be, the same.

Bacterial Proteins↗

PubMed2003

Validation of the summary ROC for diagnostic test meta-analysis: a Monte Carlo simulation.

RATIONALE AND OBJECTIVES: The author performed this study to test a technique for validating the logit regression method for summary receiver operating characteristic (ROC) meta-analysis, perform initial validation studies, and identify areas for further investigation. MATERIALS AND METHODS: Monte Carlo simulation was performed by using a custom macro program for a personal computer spreadsheet. The program creates simulated data sets based on user-specified parameters, performs a meta-analysis on the data sets, and logs the results so the accuracy and variability of the method can be measured. The program can also be used to measure the effects of changes in study design and meta-analysis parameters. RESULTS: For the base case of a small meta-analysis (10 studies) of small trials (mean, 50 patients), the meta-analysis results closely matched the input sensitivity and specificity when they were less than 80%. Systematic errors, if any, were small. At sensitivities and specificities greater than 80%, the true sensitivity or specificity was underestimated by up to 2% in the meta-analysis. Confidence intervals calculated with the summary ROC curve were reasonably conservative, although they too fell below the true results when sensitivity or specificity was greater than 80%. The underestimation was eliminated when the simulation was repeated for a much larger trial (mean, 1,000 cases per study)--even with a sensitivity and specificity of 98%. CONCLUSION: The Littenberg-Moses method for summary ROC meta-analysis is effective for obtaining an accurate summary estimate of diagnostic test performance, although the continuity correction introduces a small downward bias in the meta-analysis of small trials.

Computer Simulation↗