SEARCH · PubMed Health

Results for “roc”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5Linked to original sources

PubMed2001

ROC with confidence - a Perl program for receiver operator characteristic curves.

Receiver operator characteristic (ROC) curves are recommended to assess the diagnostic value of tests depending on a single cut-off value of a continuous variable. These ROC curves show the true-positive rate (sensitivity) against the false-positive rate (1-specificity). It is desirable, especially in situations with small samples of observations, to display confidence bounds of the ROC curve. This paper presents a Perl program which calculates the ROC curve and its distribution-free confidence bounds. A simple user interface also written in Perl permits their display.

Confidence Intervals↗

PubMed1998

Receiver operating characteristic (ROC) analysis: basic principles and applications in radiology.

Receiver operating characteristic (ROC) analysis is a widely accepted method for analyzing and comparing the diagnostic accuracy of radiological tests. In this paper we will explain the basic principles underlying ROC analysis and provide practical information on the use and interpretation of ROC curves. The major applications of ROC analysis will be discussed and their limitations will be addressed.

Humans↗

PubMed2001

Continuous versus categorical data for ROC analysis: some quantitative considerations.

RATIONALE AND OBJECTIVES: Several authors have encouraged the use of a quasi-continuous rating scale for data collection in receiver operating characteristic (ROC) curve analysis of diagnostic modalities, rather than rating scales based on five to seven ordinal categories or levels of suspicion. Although many investigators have gone over to this method, a discussion of the issues continues. The present work provides a quantitative analysis from the viewpoint of measurement science. MATERIALS AND METHODS: A simple model of the effect of data discretization or quantization on the measurement of the variance of noisy data was developed. Then Monte Carlo simulations of multiple-reader, multiple-case ROC experiments were performed and analyzed in terms of components-of-variance models to investigate the effect of data quantization in that more complex setting. RESULTS: For single-reader studies, discretization into five categories can reduce the precision of ROC measurements by a large amount. The effect may be attenuated in multireader studies. CONCLUSION: More precise measurements of diagnostic detection performance and thus more efficient use of resources are served by good measurement methods. These are promoted by the use of a quasi-continuous rating scale in ROC studies.

Data Collection↗

PubMed2002

Regularities of source recognition: ROC analysis.

Source memory has become the focus of a growing number of investigations in a variety of fields. An appropriate model for source memory is, therefore, of increasing importance. A simple 2-dimensional signal-detection model of source recognition is presented. The receiver operating characteristics (ROCs) obtained from 3 experiments are then used to test the model. The data demonstrate 3 regularities: convex ROCs, z-ROCs with linear slopes of 1.00, and slightly concave z-ROCs. Two of these regularities support the model. The 3rd requires a revision of the model. This revised model is fitted to the data. The implications of these regularities for other theories are also discussed.

Adolescent↗

PubMed2003

[Chest radiography: ROC phantom study of four different digital systems and one conventional radiographic system].

PURPOSE: To compare the diagnostic quality of five different radiographic systems used in chest radiography for visualization of differently configured, clinically relevant pathologic pulmonary structures. MATERIALS AND METHODS: Four digital detector systems using as detection unit a CsI/aSi-based flat panel detector, a transparent imaging plate, a selenium detector and a conventional storage phosphor plate were analyzed for this study, as well as an asymmetrical film-screen system. The analyzed imaging material consisted of radiographs of an anthropomorphic chest-phantom with superimposed simulated pulmonary structures. The images were evaluated for different pathologic structures by a newly developed multiple structure ROC (ms-ROC). RESULTS: The performance of each system was found to have a strong structure-related variability. The flat panel detector system had the best overall performance. The theoretical advantage of the 4k-matrix of the transparent imaging plate over the 3k-matrix of the flat panel detector was only confirmed for reticular structures. CONCLUSIONS: In addition to comparing the image quality of the different systems, this study shows that the performance of a radiographic system depends on the structure to be analyzed. The modified ROC (ms-ROC) provides valid results with less effort.

Humans↗

PubMed1998

Assessment of image quality and total performance in Norwegian mammography laboratories. Findings in a national survey based on different phantoms and ROC methodology.

PURPOSE: To assess the image quality at different mammography laboratories. MATERIAL AND METHODS: Two commercial mammographic test phantoms and one phantom based on excised mammary tissue were used in an assessment of the imaging chain and total performance at 45 Norwegian mammography laboratories. The breast-tissue phantom was used for a receiver operating characteristics (ROC) analysis. This was carried out by putting overlays with identifiable regions (some of which contained a cluster of simulated calcifications) on top of the mammary tissue, and then having a radiologist report the confidence of a finding for each region. RESULTS AND CONCLUSION: The areas under the ROC curves were in general high. In nearly all the laboratories, performance was improved when a magnification technique was applied. There were wide variations among the laboratories in total performance as measured by the area under the ROC curve, and also in the physical parameters derived by means of the commercial phantoms. In general, a good ROC performance was associated with a good physical performance in the imaging chain.

Female↗

PubMed1991

Impact of gamma camera parameters on imaging performance, evaluated by receiver operating characteristic (ROC) analysis.

The relationship between gamma camera variables (total counts in image, collimator type, etc) and diagnostic imaging performance was quantitatively investigated using receiver operating characteristic (ROC) curve analysis. A College of American Pathologists (CAP) liver phantom was used with a 99Tcm flood source to generate anterior and lateral liver images containing 'cold lesions'. These images were interpreted by four nuclear medicine physicians, and the areas under the corresponding ROC curves computed. These medicine physicians, and the areas under the corresponding ROC curves computed. These areas were taken as a quantitative estimate of the imaging performance of the system. The average area under the ROC curve for the four physicians reading the same 'standard' image six times was computed to be 66.8 +/- 5.8. Experiments were performed to show the effect on diagnostic performance of (i) increasing the total image counts from 200k to 2000k, (ii) varying the phantom-to-collimator separation from 0 to 8 cm and (iii) changing the collimator type. In all cases, data were generated which demonstrated the quantitative improvement (or deterioration) resulting from these changes. These data may be used in the design of clinical imaging protocols, for which choices have to be made for each gamma camera variable.

Gamma Cameras↗

PubMed2004

Semi-parametric ROC regression analysis with placement values.

Advances in technology provide new diagnostic tests for early detection of disease. Frequently, these tests have continuous outcomes. One popular method to summarize the accuracy of such a test is the Receiver Operating Characteristic (ROC) curve. Methods for estimating ROC curves have long been available. To examine covariate effects, Pepe (1997, 2000) and Alonzo and Pepe (2002) proposed distribution-free approaches based on a parametric regression model for the ROC curve. Cai and Pepe (2002) extended the parametric ROC regression model by allowing an arbitrary non-parametric baseline function. In this paper, while we follow the same semi-parametric setting as in that paper, we highlight a new estimator that offers several improvements over the earlier work: superior efficiency, the ability to estimate the covariate effects without estimating the non-parametric baseline function and easy implementation with standard software. The methodology is applied to a case control dataset where we evaluate the accuracy of the prostate-specific antigen as a biomarker for early detection of prostate cancer. Simulation studies suggest that the new estimator under the semi-parametric model, while always being more robust, has efficiency that is comparable to or better than the Alonzo and Pepe (2002) estimator from the parametric model.

Case-Control Studies↗

PubMed2004

Semiparametric estimation of time-dependent ROC curves for longitudinal marker data.

One approach to evaluating the strength of association between a longitudinal marker process and a key clinical event time is through predictive regression methods such as a time-dependent covariate hazard model. For example, a Cox model with time-varying covariates specifies the instantaneous risk of the event as a function of the time-varying marker and additional covariates. In this manuscript we explore a second complementary approach which characterizes the distribution of the marker as a function of both the measurement time and the ultimate event time. Our goal is to extend the standard diagnostic accuracy concepts of sensitivity and specificity so as to recognize explicitly both the timing of the marker measurement and the timing of disease. The accuracy of a longitudinal marker can be fully characterized using time-dependent receiver operating characteristic (ROC) curves. We detail a semiparametric estimation method for time-dependent ROC curves that adopts a regression quantile approach for longitudinal data introduced by Heagerty and Pepe (1999, Applied Statistics, 48, 533-551). We extend the work of Heagerty and Pepe (1999, Applied Statistics, 48, 533-551) by developing asymptotic distribution theory for the ROC estimators where the distributional shape for the marker is allowed to depend on covariates. To illustrate our method, we analyze pulmonary function measurements among cystic fibrosis subjects and estimate ROC curves that assess how well the pulmonary function measurement can distinguish subjects that progress to death from subjects that remain alive. Comparing the results from our semiparametric analysis to a fully parametric method discussed by Etzioni et al. (1999, Medical Decision Making, 19, 242-251) suggests that the ability to relax distributional assumptions may be important in practice.

Adolescent↗

PubMed1992

ROC curves for the initial assessment of new diagnostic tests.

New diagnostic tests are mainly evaluated by determining the sensitivity and specificity of the test. These test characteristics were originally meant to be used in making diagnoses. For evaluative purposes their usefulness is weakened by their susceptibility to selection and their dependence on the cut-off points that are used for test positivity. The plotting of a receiver operating characteristic (ROC) curve might be a solution to these problems. Furthermore, the ROC curve yields a measure for the diagnostic power of the test expressed in one number instead of two, namely the area under the curve (AUC). Finally, the ROC curve and its AUC permit easy comparison of different tests and the performance of different interpreters of one test. The construction and use of ROC curves are described and illustrated with data of a case-referent investigation into the relationship between iron status parameters and the presence of acute myocardial infarction. The AUCs of ferritin and serum iron, 0.61 and 0.68 respectively, are too low to suggest meaningful usefulness in clinical practice.

Humans↗

PubMed2003

On the equivalence of classic ROC analysis and the loss-function model to set cut points in sequential testing.

In an effort to reduce the cost of administration for objective structured clinical examinations (OSCEs), several authors have promoted the use of sequential testing in which all candidates take a short screening test and candidates who pass the screen are exempted from taking the full test. Traditionally, the determination of the optimally efficient cut point (passing score) for the screen has used ROC analysis to minimize false-positive and false-negative errors. Recently, Muijtjens et al. have questioned the appropriateness of the ROC method for these purposes and have promoted an alternative method that uses a "loss" formula. However, given certain theoretically derived conditions, it can be shown that the use of the loss formula is functionally identical to using ROC analysis, and the authors suggest that continued use of the ROC method is appropriate.

Education, Medical↗

PubMed1989

Some practical issues of experimental design and data analysis in radiological ROC studies.

Receiver operating characteristic (ROC) analysis has been used in a broad variety of medical imaging studies during the past 15 years, and its advantages over more traditional measures of diagnostic performance are now clearly established. But despite the essential simplicity of the approach, workers in the field often find--sometimes only after an ROC study is under way--that a number of subtle issues related to experimental design and data analysis must be confronted in practice. Many of these issues have not been discussed in the literature in detail, and most are not well known. The purposes of this paper are to make users of ROC methodology in medical imaging aware of potential problems that should be confronted before an ROC study is begun and to indicate, at least broadly, how those problems may be dealt with, given the present state of the art. Some of the issues raised here can be addressed adequately by easily prescribed techniques, whereas others remain difficult and will be resolved fully only by new methodologic developments.

ROC Curve↗

PubMed1990

Effect of observer instruction on ROC study of chest images.

Receiver-operating characteristic (ROC) analysis has been used in many medical imaging applications during the past decade. In order to ensure that reader-confidence ratings are analyzable (well distributed to meet convergence requirements of curve-fitting algorithms) and meaningful (limit extrapolation of the data), many investigators train readers specifically for this purpose. No experimental data are available concerning the possible effects of such training on the results of ROC studies. We performed a multi-observer, multi-disease study in which 300 chest images were rated by four radiologists before and after they were trained to provide well-distributed confidence ratings. The results indicate that for our data set, reader and disease-specific accuracy was not significantly affected by the training process for interstitial disease and pneumothoraces. However, the accuracy of two readers was significantly affected for the detection of nodules (P less than 0.05), and the overall accuracy of one reader was significantly affected for the classification of normal versus abnormal images (P less than 0.01). Thus, in spite of the difficulties associated with the performance of ROC studies in a free-reading environment, one should carefully consider the possible effects of any intervention on the results prior to conducting ROC studies.

Humans↗

PubMed2000

An interpretation for the ROC curve and inference using GLM procedures.

The accuracy of a medical diagnostic test is often summarized in a receiver operating characteristic (ROC) curve. This paper puts forth an interpretation for each point on the ROC curve as being a conditional probability of a test result from a random diseased subject exceeding that from a random nondiseased subject. This interpretation gives rise to new methods for making inference about ROC curves. It is shown that inference can be achieved with binary regression techniques applied to indicator variables constructed from pairs of test results, one component of the pair being from a diseased subject and the other from a nondiseased subject. Within the generalized linear model (GLM) binary regression framework, ROC curves can be estimated, and we highlight a new semiparametric estimator. Covariate effects can also be evaluated with the GLM models. The methodology is applied to a pancreatic cancer dataset where we use the regression framework to compare two different serum biomarkers. Asymptotic distribution theory is developed to facilitate inference and to provide insight into factors influencing variability of estimated model parameters.

Humans↗

PubMed2000

ROC curve estimation when covariates affect the verification process.

A receiver operating characteristic (ROC) curve is commonly used to measure the accuracy of a medical test. It is a plot of the true positive fraction (sensitivity) against the false positive fraction (1-specificity) for increasingly stringent positivity criterion. Bias can occur in estimation of an ROC curve if only some of the tested patients are selected for disease verification and if analysis is restricted only to the verified cases. This bias is known as verification bias. In this paper, we address the problem of correcting for verification bias in estimation of an ROC curve when the verification process and efficacy of the diagnostic test depend on covariates. Our method applies the EM algorithm to ordinal regression models to derive ML estimates for ROC curves as a function of covariates, adjusted for covariates affecting the likelihood of being verified. Asymptotic variance estimates are obtained using the observed information matrix of the observed data. These estimates are derived under the missing-at-random assumption, which means that selection for disease verification depends only on the observed data, i.e., the test result and the observed covariates. We also address the issues of model selection and model checking. Finally, we illustrate the proposed method on data from a two-phase study of dementia disorders, where selection for verification depends on the screening test result and age.

Aged↗

PubMed1992

Variation in restorative treatment decisions: application of Receiver Operating Characteristic curve (ROC) analysis.

It has been evident for many years that dentists, when planning treatment for patients, do not act in a standard manner, and previous research has shown there to be wide variations in treatment planning amongst groups of dentists. Signal detection theory and Receiver Operating Characteristic (ROC) analysis allows measurement of an observer's ability to detect a lesion, while at the same time allowing examination of how a lesion, once perceived, is judged to be in need of treatment. An ROC curve is constructed by plotting the sensitivity (or true positive rate) of decisions made, against the false positive rate (equivalent to 1-specificity) when various decision attitudes, from interventionist to non-interventionist, are held. Fifteen pairs of simulated bitewing radiographs were shown to 20 dentists, who were asked to specify, for each approximal lesion, whether or not they would place a conventional restoration. The 7200 decisions made by the dentists were validated by sectioning and microscopically examining the teeth. The mean sensitivity of the dentists' decisions, when the strictest operating thresholds were held and caries into dentine was the validating criterion, was 0.26 and the mean specificity was 0.96. ROC analysis shows that when operating at the strictest threshold, the dentists were implying that specificity was weighted as being 2.7 times more important than sensitivity. ROC analysis leads to insight into how dentists differentially weight the true and false, positive and negative, outcomes of their decisions and thus allows explanation of why two dentists would rarely make exactly the same treatment plan for one patient, and also why different treatments might be offered to two patients exhibiting the same levels of disease.

Decision Support Techniques↗

PubMed1993

Applicability of Receiver Operating Characteristic (ROC) analysis on discrete caries depth ratings.

Accuracy of diagnostic tests for caries detection is frequently evaluated by sensitivity and specificity values obtained from data acquired on an ordinal rating scale which classifies lesions according to the depth in enamel and dentin. The interpretation of these parameters may, however, be difficult. Receiver Operating Characteristic (ROC) analysis, on the other hand, yields an expression for overall diagnostic accuracy in a single parameter, facilitating a comparison of performance of several diagnostic tests. The basis for applying ROC analysis has been that data were collected on a confidence (likelihood) scale. The aim of this preliminary study was to investigate whether the application of ROC analysis would yield as valid parameters for diagnostic accuracy in assessment of approximal caries on a depth rating scale as on a likelihood rating scale. The results demonstrated that the accuracy of observer performance was not influenced by the type of rating scale employed, justifying the application of ROC analysis on discrete caries depth ratings.

Chi-Square Distribution↗

PubMed2025

Functional and Pangenomic Exploration of Roc Two-Component Regulatory Systems Identifies Novel Players Across Pseudomonas Species.

The opportunistic pathogen Pseudomonas aeruginosa relies on a large collection of two-component regulatory systems (TCSs) to sense and adapt to changing environments. Among them, the Roc (regulation of cup) system is a one-of-a-kind network of branched TCSs, composed of two histidine kinases (HKs-RocS1 and RocS2) interacting with three response regulators (RRs-RocA1, RocR, and RocA2), which regulate virulence, antibiotic resistance, and biofilm formation. Based on extensive work on the Roc system, previous data suggested the existence of other key regulators yet to be discovered. In this work, we identified PA4080, renamed RocA3, as a fourth RR that is activated by RocS1 and RocS2 and that positively controls the expression of the cupB operon. Comparative genomic analysis of the locus identified a gene-rocR3-adjacent to rocA3 in a subpopulation of strains that encodes a protein with structural and functional similarity to the c-di-GMP phosphodiesterase RocR. Furthermore, we identified a fourth branch of the Roc system consisting of the PA2583 HK, renamed RocS4, and the Hpt protein HptA. Using a bacterial two-hybrid system, we showed that RocS4 interacts with HptA, which in turn interacts with RocA1, RocA2, and RocR3. Finally, we mapped the pangenomic RRs repertoire, establishing a comprehensive view of the plasticity of such regulators among clades of the species. Overall, our work provides a comprehensive inter-species definition of the Roc system, nearly doubling the number of proteins known to be involved in this interconnected network of TCSs controlling pathogenicity in Pseudomonas species.

Gene Expression Regulation, Bacterial↗