A contaminated binormal model for ROC data: Part III. Initial evaluation with detection ROC data.

Abstract

RATIONALE AND OBJECTIVES: The authors' purpose was to evaluate how well the contaminated binormal receiver operating characteristic (ROC) model fits (a) degenerate data for which standard ROC models commonly fail and (b) nondegenerate data from exemplary experiments, for which the standard binormal model should be appropriate. MATERIALS AND METHODS: The authors studied two examples of binormally degenerate data, with and without interior points, and ROC rating data from four experiments in visual psychophysics and radiology. The plots of contaminated binormal ROC curves of the binormal degenerate data were examined. For ROC data with at least one interior point, the new model was compared with conventional models on the basis of likelihood-ratio chi2 statistics (G2). RESULTS: With no interior points, the contaminated binormal model gave results consistent with the fundamental principle underlying ROC analysis, that is, for a fixed false-positive probability, the higher the true-positive probability, the better the diagnostic performance. Contaminated binormal ROC curves go through the empirical ROC points of the degenerate data without crossing the chance line or climbing far above the true-positive fractions of the points. For several model ROC studies, the contaminated binormal model gave smaller G2 results than conventional ROC models, although the differences tended to be small, usually with little difference in ROC area. CONCLUSION: The contaminated binormal model fits binormal degenerate data better than conventional ROC models, and it offers an explanation for the degeneracy. The lower G2 values on some classic, nondegenerate ROC data suggest that contamination may not be limited to degenerate ROC data.