SEARCH · PubMed Health

Results for “roc”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7Linked to original sources

PubMed1995

ROC curves with multiple marker measurements.

Properties of receiver operating characteristic (ROC) curves are explored for markers that are measured repeatedly, through space or time, for each subject. The true underlying response, positive or negative, of each subject is assumed to be constant across marker measurements, and is determined from assessment of some "gold standard." A marker-based test is considered positive when at least one of a subject's marker values exceeds a designated cutoff. If subjects with positive and negative underlying responses differ in the number of marker measurements per subject, the ROC curve for a noninformative marker is bowed above or below the diagonal line representing the "null" curve for a marker that is measured just once per subject. If subjects with negative responses tend to have more measurements than those with positive responses, the ROC curve for even an informative marker may lie beneath the curve for the same marker measured once per subject. The form of the ROC curve for a marker used in this way is strongly influenced by the strength of the correlation of measurements within subjects.

Biomarkers↗

PubMed1995

Evaluation of partial classification algorithms using ROC curves.

When using computer programs for decision support in clinical routine, an assessment or a comparison of the underlying classification algorithms is essential. In classical (forced) classification, the classification rule always selects exactly one alternative. A number of proven discriminant measures are available here, e.g.sensitivity and error rate. For probabilistic classification, a series of additional measures has been developed [1]. However, for many clinical applications, there are models where an observation is classified into several classes (partial classification), e.g., models from artificial intelligence, decision analysis, or fuzzy set theory. In partial classification, the discriminatory ability (Murphy) can be adjusted a priori to any level, in most practical cases. Here the usual measures do not apply. We investigate the preconditions for assessment and comparison based on medical decision theory. We focus on problems in the medical domain and establish a methodological framework. When using partial classification procedures, a ROC analysis in the classical sense is no longer appropriate. In forced classification for two classes, the problem is to find a cutoff point on the ROC curve; while in partial classification, you have to find two of them. They characterize the elements being classified as coming from both classes. This extends to several classes. We propose measures corresponding to the usual discriminant measures for forced classification (e.g., sensitivity and error rate) and demonstrate the effects using the ROC approach. For this purpose, we extend the existing method for forced classification in a mathematically sound manner. Algorithms for the construction of thresholds can easily be adapted. Two specific measurement models, based on parametric and non-parametric approaches, will be introduced. The basic methodology is suitable for all partial classification problems, whereas the extended ROC analysis assumes a rank order of the selected alternatives. The method is based on the black box principle and makes use only of the results of the compared algorithms and some general intuitive principles. Therefore, algorithms arising from different "philosophical" approaches may also be compared according to their results. For demonstration purposes, a clinical example from cranial computed tomography is presented. Linear discriminant analysis as a reference is compared to a Bayesian and a fuzzy procedure.

Algorithms↗

PubMed1997

Confidence intervals for the generalized ROC criterion.

Receiver operating characteristic (ROC) curves are frequently used to assess the usefulness of diagnostic markers. When several diagnostic markers are available, they can be combined by a best linear combination: that is, when the area under the ROC curve of this combination is maximized among all possible linear combinations. This maximal area is the generalized ROC criterion, which provides a measure of how effective the combination of the markers is. This criterion needs to be estimated from the data, and is usually evaluated against single markers. In the present paper, we provide confidence intervals for the generalized ROC criterion under the assumption of homogeneous covariance matrices, derive an approximation for the heterogeneous covariance matrices case, and evaluate the approximation via a simulation study. Finally, we present an illustrative example.

Biomarkers↗

PubMed1995

A modified ROC analysis for the selection of cut-off values and the definition of intermediate results of serodiagnostic tests.

A total number of 50 sera from clinically confirmed cases of canine Borrelia (B.) burgdorferi infection and 44 negative control sera were tested with a B. burgdorferi specific antibody ELISA. The data were submitted to the 'two-graph receiver operating characteristic' (TG-ROC) analysis which is a plot of the test sensitivity (Se) and specificity (Sp) against the threshold (cut-off) value assuming the latter to be an independent variable. Thus, in contrast to the conventional ROC analysis, valid pairs of Se and Sp can be read for pre-assigned threshold values directly from the TG-ROC plots. A cut-off that realises equal test parameters (Se = Sp = theta 0 (theta-zero)) can be obtained as the intersection point of the two graphs. Since the value for theta 0 is below a preselected accuracy level (95% or 90%), two cut-off values are selected that represent the bounds of an 'intermediate range' (IR). IR can be considered as a 'borderline' range for the clinical interpretation of test results. The proportion of the measurement range (MR) that gives unambiguous test results can be expressed using IR as the 'valid range proportion' (VRP = (MR-IR)/MR). VRP and theta 0 are useful parameters for test comparison since they do not depend upon the selection of a single cut-off point. In addition, the selection of cut-off values is supported by graphical displays of efficiency, Youden's index and likelihood ratios which can be considered as functions of the pre-assigned cut-off value. TG-ROC was derived as a user-defined template for a commercially available spreadsheet programme (MS-EXCEL, Microsoft).

Animals↗

PubMed2005

Rat epidermal keratinocyte organotypic culture (ROC) as a model for chemically induced skin irritation testing.

The potential of rat epidermal keratinocyte (REK) organotypic culture (ROC) with proper stratum corneum barrier as a model for screening skin irritants was evaluated. The test chemicals were selected from ECETOC database (1995) and the observed in vitro irritation potential was compared to ECETOC in vivo primary irritation index (PII), to EU risk phrases, and to the harmonized OECD criteria. Chemicals were applied onto the stratum corneum surface of ROC for 30 min and samples were taken from the underlying medium at 4 and 8 h after exposure. Cell membrane integrity (determined by LDH assay) and pro-inflammatory effect (determined by IL-1alpha release) were verified at both time points and correlated to PII values. The best correlation (R(2) = 0.831) was seen with LDH leakage test. Based on obtained data, chemicals were classified according to criteria defined by EU and OECD. From 12 chemicals, only two were incorrectly classified according to OECD criteria when using LDH leakage and IL-1alpha release as irritation markers. At the end of experiment, chemical-treated ROC cultures were fixed and histological changes were assessed. Typical signs for irritation were lightly stained cytoplasm, condensed nuclei, cellular vacuolization, eosinophilic cytoplasms, and blebbing. These irritation effects of chemicals were graded visually into four classes (A-D). The extent of morphological perturbations of the cultures mostly correlated with PII. The present results indicate the validity of the ROC model in predicting skin irritation potential of chemicals and show that the use of set of irritation markers with different mechanistic responses gives more information on irritation than if only one marker was used.

Animals↗

PubMed1985

A comparison of the validity of two psychiatric screening questionnaires (GHQ-12 and SRQ-20) in Brazil, using Relative Operating Characteristic (ROC) analysis.

The General Health Questionnaire (GHQ-12) (Goldberg, 1972) and the Self Report Questionnaire (SRQ-20) (Harding et al. 1980) were simultaneously validated against the criterion of the Clinical Interview Schedule (CIS) (Goldberg et al. 1970) in three primary care clinics in the city of Sao Paulo, Brazil. A comparison between the two screening instruments was carried out. The product-moment correlation between the two sets of scores was +0.72, and the validation coefficients for the GHQ-12 and the SRQ-20 were respectively: sensitivity 85% and 83%; specificity 79% and 80%; overall misclassification rate 18% and 19%. The two screening instruments were further compared by the application of Relative Operating Characteristic (ROC) analysis and, again, were found to be very similar in performance. The area under the ROC curve was 0.87 for the GHQ and 0.90 for the SRQ-20 (Z = 0.92, NS). Both psychiatric screening instruments were therefore found to be acceptably valid when assessed against the CIS in three primary care settings in Brazil. The application of ROC analysis to studies of this kind was feasible and straightforward. It was found to be superior to the conventional method of presentation of validity data. It is recommended that the application of ROC analysis to psychiatric screening studies be further utilized and explored.

Brazil↗

PubMed2002

Distribution-free ROC analysis using binary regression techniques.

Receiver operating characteristic (ROC) regression methodology is used to identify factors that affect the accuracy of medical diagnostic tests. In this paper, we consider a ROC model for which the ROC curve is a parametric function of covariates but distributions of the diagnostic test results are not specified. Covariates can be either common to all subjects or specific to those with disease. We propose a new estimation procedure based on binary indicators defined by the test result for a diseased subject exceeding various specified quantiles of the distribution of test results from non-diseased subjects with the same covariate values. This procedure is conceptually and computationally simplified relative to existing procedures. Simulation study results indicate that the approach has fairly high statistical efficiency. The new ROC regression methodology is used to evaluate childhood measurements of body mass index as a predictive marker of adult obesity.

Journal Article↗

PubMed1986

Digital and conventional chest imaging: a modified ROC study of observer performance using simulated nodules.

A modified receiver operating characteristic (ROC) study was performed in which five readers were asked to locate multiple nodules on images of an anthropomorphic phantom obtained with a prototype digital radiographic chest unit and with a conventional chest unit. Results indicate that when nodules were projected over the lungs, a significantly greater number (significant at the 5% level) were identified on conventional radiographs, whereas for nodules projected over the mediastinum, the digital images were notably superior (difference significant at the 2% level). An error analysis of the multiple nodule problem and pseudo-ROC curves are presented. The modified ROC study does not suffer from the positional ambiguity inherent in most ROC studies and is efficient in acquiring data.

Analog-Digital Conversion↗

PubMedDate not supplied

[Use of ROC analysis in psychiatry].

The fundamental principles of ROC analysis are described. This method provides a means to assess the overall discriminant power of psychiatric rating scales for the full range of their scores. For each cut-off, an instrument has a sensitivity (true positive rate) and a specificity (true negative rate). High values of these coefficients are desirable although they are inversely related. ROC curves can be obtained by plotting the false positive rate and the true positive rate for different thresholds of the rating scale. The curves which would be obtained with a perfect, a worthless and a typical instrument are drawn to illustrate various situations found in ROC analysis. Among the several indices proposed, the area under the curve (AUC) is the most commonly used index to assess the overall discriminant power of an instrument. To calculate this area, the non parametric trapezoidal method is advocated. The area under the curve varies between 0.50 which corresponds to the chance line up to 1.0, a value associated with perfect accuracy. This parameter can be interpreted as the probability of classifying correctly the subjects of a pair where one is normal and one is diseased. Then, the appropriate statistic to compare several ROC indices is provided for the general case of independent observations. When the observations are paired, the standard error of the difference between two areas needs to be corrected.(ABSTRACT TRUNCATED AT 250 WORDS)

Child↗

PubMed1985

The use of ROC curves in histopathologic decision making.

The applicability of receiver operator characteristic (ROC) curves to histopathology is presented in three examples, along with a basic discussion of some properties of ROC curves. One major application is in the attempt to define stages in diseases that show a continuous spectrum of histologic patterns, in which the uncertainty of boundary points and the overlap of features makes such definition difficult. Construction and analysis of ROC curves may help to identify the features with the greatest utility, as, for example, in the grading of mucinous carcinomas of the ovary. ROC curves can also be used to assess diagnostic differences between histopathologists, whether they are using different criteria or the same criteria but with different weightings, as, for example, in cervical premalignancy or borderline ovarian tumors.

Adenocarcinoma, Mucinous↗

PubMed1986

The use of relative operating characteristic (ROC) curves in test performance evaluation.

Relative (or receiver) operating characteristic (ROC) curves are a graphical representation of the relationship between sensitivity and specificity of a laboratory test over all possible diagnostic cutoff values. Laboratory medicine has been slow to adopt the use of ROC curves for the analysis of diagnostic test performance. In this tutorial, we discuss the advantages and limitations of the ROC curve for clinical decision making in laboratory medicine. We demonstrate the construction and statistical uses of ROC analysis, review its published applications in clinical pathology, and comment on its role in the decision analytic framework in laboratory medicine.

Clinical Laboratory Techniques↗

PubMed1995

[Studies and recommendations on the design ov ROC analyses in nuclear medicine].

ROC analysis is the method of choice for an objective assessment of diagnostic tests; however, it requires large sample sizes. We investigated the influence of study design and data analysis on sampling requirements. A sample size of 123 for the patient as well as the control group, is required to prove a difference in sensitivity of 75% vs 90% (specificity 90%, statistical power 0.8). Analysis with the binormal bivariate ROC model allows > 35% reduction in sample size. If the patient group is increased the control group can be smaller, and vice versa; here the correlated ROC model also allows substantial decreases in sample size. If both diagnostic tests are performed in the same patient and control group and evaluated with the correlated ROC model, an objective, statistically sound assessment of diagnostic performance is possible with relatively small samples.

Analysis of Variance↗

PubMed1996

A measure of association for assessing prediction accuracy that is a generalization of non-parametric ROC area.

There is a need for a measure of prediction accuracy that generalizes non-parametric receiver operating characteristic (ROC) area to polytomous ordinal patient state. We describe such a measure, prediction probability PK derived from Kim's measure of association. We show that the value of PK equals the value of non-parametric ROC area for dichotomous patient state and is a meaningful generalization of non-parametric ROC area for polytomous state.

Anesthesia↗

PubMed1996

The use of the 'binormal' model for parametric ROC analysis of quantitative diagnostic tests.

The binormal model is widely used for parametric receiver operating characteristic (ROC) analyses of data concerning the accuracy of medical diagnostic tests. Empirical evaluation of the performance of this model in the face of departures from binormality has been limited to interpretations of radiology-type examinations recorded on a rating scale. This paper extends the investigation to the performance of the model with biochemical and other tests recorded on an interval scale. In order to describe non-binormal pairs of distributions, a useful standardized graphical display is developed; this display also illustrates several features of ROC curves. We consider non-binormal pairs of distributions with or without a monotone likelihood ratio and show that by transformation of the underlying scale, one can make many such pairs resemble closely the binormal model. These findings justify Metz's use of the binormal model in the 'LABROC' software for ROC analyses of laboratory type data even when the raw data may 'look' decidedly non-Gaussian.

Diagnostic Imaging↗

PubMed1996

Waist circumference as a screening tool for cardiovascular risk factors: evaluation of receiver operating characteristics (ROC).

UNLABELLED: To evaluate the receiver operating characteristics (ROC) to determine the cutoffs of waist circumference as a potential population directed screening tool for hypercholesterolaemia (> or = 6.5 mmol/L), low high density lipoprotein cholesterol (< 0.9 mmol/L), and hypertension (treated and/or systolic > or = 160 and/or diastolic blood pressure > or = 95 mmHg), in 2183 men and 2698 women aged 20 to 59 years selected at random from Dutch civil registries. MAIN OUTCOME MEASURES: Height, weight, body mass index (BMI), waist circumference, total plasma cholesterol and high density lipoprotein cholesterol concentrations, and blood pressure. RESULTS: ROC curves showed that sensitivity equalled specificity at waist circumferences between 93-95 cm in men and 81-84 cm in women for identifying individual risk factors, and 92 cm in men and 81 cm in women for identifying those with at least one risk factor. Sensitivity and specificity were equal at levels between 61% to 69% for identifying individual risk factors, with positive predictions (56.8% in men and 37.8% in women) within 2% of those using previously defined 'Action Level 1' of waist circumference 94 cm in men and 80 cm in women (58.8% in men and 37.4% in women). Risk prediction by anthropometric methods was relatively low: ROC areas for identifying each risk factor by waist varied from 55% to 60%, and reached about 65% for identifying at least one risk factor. Height accounted for less than 0.3% of variance in waist circumference. Using BMI at 25 kg/m2 gave similar prediction to waist, but its combination with waist did not improve predictive values. CONCLUSIONS: Measurement of waist circumference 'Action Level 1' at 94 cm (37 inches) in men and 80 cm (32 inches) in women could be adopted as a simpler valid alternative to BMI for health promotion, to alert those at risk of cardiovascular disease, and as a guide to risk avoidance by self-weight management.

Adult↗

PubMed1995

An ROC approach for evaluating functional brain MR imaging and postprocessing protocols.

A method that can be used to evaluate the performance of MRI methods for detecting discrete regional activations using functional MRI is presented. Computer derived receiver-operator-characteristic (ROC) curves have been used to evaluate quantitatively a range of conditions encountered in functional MRI studies. ROC analysis allows multiple acquisition strategies and multiple postprocessing strategies to be quantitatively and objectively compared. The authors first present this analysis technique and then illustrate its use for assessing the relative performances of different functional MRI data acquisition strategies using different gradient echo, echoplanar imaging protocols. In addition, the authors have used the ROC analysis to evaluate and compare several methods for analyzing functional MRI data to extract regions of activation. This approach to assessing the performance of different methods is of general use and can be applied to evaluate other data acquisition protocols and postprocessing methods.

Algorithms↗

PubMed2002

Comparison of bandwidth selection methods for kernel smoothing of ROC curves.

In this paper we compared four non-parametric kernel smoothing methods for estimating an ROC curve based on a continuous-scale test. All four methods produced a smooth ROC curve of the test. The difference in these four methods lay with the way they chose their bandwidth parameters. To assess the relative performance of the four bandwidth selection methods, we conducted a simulation study using different underlying distributions, along with varied sample sizes. The results from our simulation study suggested that the kernel smoothing method originally proposed by Altman and Léger for estimation of the distribution function was the best choice for estimation of an ROC curve. We illustrated these methods with a real example.

Computer Simulation↗

PubMed2003

ROC curve analysis for biomarkers based on pooled assessments.

Interleukin-6 is a biomarker of inflammation which has been suggested as having potential discriminatory ability for myocardial infarction. Because of its high assaying cost it is very expensive to evaluate this marker. In order to reduce this cost we propose pooling the specimens. In this paper we examine the efficiency of ROC curve analysis, specifically the estimation of the area under the ROC curve, when dealing with pooled data. We study the effect of pooling when there are only a fixed number of individuals available for testing and pooling is carried out to save on the number of assays. Alternatively we examine how many pooled assays of size g are necessary to provide essentially the same information as N individual assays. We measure loss of information by means of the change in root mean square error of the estimate of the area under the ROC curve and study the extent of this loss via a simulation study.

Biomarkers↗