PubMed Health⌕ Search

Biomedical subjects

Dev P Chakraborty

Publications and source records attributed to Dev P Chakraborty.

7 recordsLinked to original sources
PubMed2006

Analysis of location specific observer performance data: validated extensions of the jackknife free-response (JAFROC) method.

RATIONALE AND OBJECTIVES: The free-response paradigm is being increasingly used in the assessment of medical imaging systems. The currently implemented method of analyzing the data, namely jackknife free-response (JAFROC) analysis, has some validation and applicability limitations. The purpose of this work is to address these limitations. MATERIALS AND METHODS: The general principles of modality evaluation and methodology validation are reviewed. A model for simulating free-response data was used to test the statistical validity of several methods of analyzing the data. The methods differed only in the choice of the figure of merit used to quantify performance. Statistical validity was judged by investigating the behaviors of the methods under null hypothesis conditions of no difference between modalities. RESULTS: The validity of the different methods of analyzing the data was found to be dependent on the choice of figure of merit. A figure of merit is identified that accommodates abnormal images with multiple (one or more) lesions, detections of which could have different clinical significances (weights). This figure of merit is shown to be statistically valid. An extension of the analysis to single reader interpretations of images from different modalities is also shown to be statistically valid. CONCLUSION: With the validated enhancements, JAFROC is expected to be of greater utility to users of the free-response method. The extension to single-reader interpretations should be of particular value to developers of image processing algorithms, including developers of computer-aided diagnosis algorithms.

Algorithms↗
PubMed2005

Recent advances in observer performance methodology: jackknife free-response ROC (JAFROC).

The jackknife free-response receiver operating characteristic (JAFROC) method allows quantitative analysis of observer data such as that observed when radiologists interpret images, which could contain more than one lesion and a location can be reported for each perceived lesion. The method was recently validated with a perception-based simulation model that incorporated the detectability parameter of the standard binormal ROC model, and in addition allowed simultaneous samples from both noise and signal distributions. The total number of noise samples is an important new parameter that measures reader expertise. The new sampling model incorporates search, which is an integral part of lesion detection that has not been possible to model until now. The model was used to generate simulated FROC ratings data, which was used to assess the statistical validity of JAFROC analysis. We found that JAFROC analysis is a statistically valid approach for analysing FROC data and that JAFROC analysis exhibited significantly greater statistical power than the existing ROC approach.

Algorithms↗
PubMed2005

A comparison of two data analyses from two observer performance studies using Jackknife ROC and JAFROC.

The authors compared two methodological approaches, Jackknife ROC and JAFROC, in analyzing data ascertained during FROC (free-response receiver operating characteristics) type studies. Observer rating data obtained from two observer performance studies were analyzed. During the first study, seven radiologists interpreted 120 mammography examinations depicting 57 masses under five different conditions with and without the results of computer-aided detection (CAD). In the second study, eight radiologists interpreted 110 examinations depicting 51 masses under six different display conditions with and without CAD results. Readers rated the detection task in a FROC type response. Jackknife ROC (using the software of LABMRMC with the highest rating per case) and JAFROC were used to compute differences, if any, in summary performance levels among all reading modes in each study as well as for all paired data sets. The results of the different analytical approaches are compared. The overall results for all modes were significantly different for the first study (p < 0.05) and not significant (p > 0.05) for the second study using either analytical approach. In the first study, the performance levels represented by three paired data sets were significantly different (p < 0.05) when computed using LABMRMC and four pairs were significantly different (p < 0.05) using JAFROC. In eight of ten pairs, JAFROC produced lower p values than LABMRMC. In the second study, LABMRMC showed no significant differences for any paired data sets and JAFROC showed a significant difference for one pair. In 15 of 16 pairs, p values computed by JAFROC were lower than those computed by LABMRMC.

Breast Neoplasms↗
PubMed2005

Free-response receiver operating characteristic evaluation of lossy JPEG2000 and object-based set partitioning in hierarchical trees compression of digitized mammograms.

PURPOSE: To assess the effects of two irreversible wavelet-based compression algorithms--Joint Photographic Experts Group (JPEG) 2000 and object-based set partitioning in hierarchical trees (SPIHT)--on the detection of clusters of microcalcifications and masses on digitized mammograms. MATERIALS AND METHODS: The use of the images in this retrospective image-collection study was approved by the institutional review board, and patient informed consent was not required. One hundred twelve mammographic images (28 with one or two clusters of microcalcifications, 19 with one mass, 17 with both abnormal findings, and 48 with normal findings) obtained in 60 women who ranged in age from 25 to 79 years were digitized and compressed at 40:1 and 80:1 by using the JPEG2000 and object-based SPIHT methods. Five experienced radiologists were asked to locate and rate clusters of microcalcifications and masses on the original and compressed images in a free-response receiver operating characteristic (FROC) data acquisition paradigm. Observer performance was evaluated with the jackknife FROC method. RESULTS: The mean FROC figures of merit for detecting clusters of microcalcifications, masses, and both radiographic findings on uncompressed images were 0.80, 0.81, and 0.72, respectively. With object-based SPIHT 80:1 compression, the corresponding values were larger than the values for uncompressed images by 0.005, 0.009, and -0.005, respectively. The 95% confidence interval for the differences in figures of merit between compressed and uncompressed images was -0.039, 0.033 for the microcalcification finding; -0.055, 0.034 for the mass finding; and -0.039, 0.030 for both findings. Because each of these confidence intervals includes zero, no significant difference in detection accuracy between uncompressed and object-based SPIHT 80:1 compression was observed at a P value of 5%. The F test of the null hypothesis that all of the modes (uncompressed and four compressed modes) were equivalent yielded the following results: F = 0.255, P = .903 for the microcalcification finding; F = 0.340, P = .848 for the mass finding; and F = 0.122, P = .975 for both findings. CONCLUSION: To within the accuracy of these measurements, lossy compression of digital mammographic data at 80:1 with JPEG2000 or the object-based SPIHT algorithm can be performed without decreasing the rate of detection of clusters of microcalcifications and masses.

Adult↗
PubMed2004

Observer studies involving detection and localization: modeling, analysis, and validation.

Although the receiver operating characteristic (ROC) paradigm is the accepted method for evaluation of diagnostic imaging systems, it has some serious shortcomings inasmuch as it is restricted to one observer report per image. By contrast the free-response ROC (FROC) paradigm and associated analysis method allows the observer to report multiple abnormalities within each imaging study, and uses the location of reported abnormalities to improve the measurement. Because the ROC method cannot accommodate multiple responses or use location information, its statistical power will suffer. The FROC paradigm/analysis has not enjoyed widespread acceptance because of concern about whether responses made to the same diagnostic study can be treated as independent. We propose a new jackknife FROC analysis method (JAFROC) that does not make the independence assumption. The new analysis method combines elements of FROC and the Dorfman-Berbaum-Metz (DBM) methods. To compare JAFROC to an earlier free-response analysis method (specifically the alternative free-response, or AFROC method), and to the DBM method, which uses conventional ROC scoring, we developed a model for generating simulated FROC data. The simulation model is based on an eye-movement model of how experts evaluate images. It allowed us to examine null hypothesis (NH) behavior and statistical power of the different methods. We found that AFROC analysis did not pass the NH test, being unduly conservative. Both the JAFROC method and the DBM method passed the NH test, but JAFROC had more statistical power than the DBM method. The results of this comparison suggest that future studies of diagnostic performance may enjoy improved statistical power or reduced sample size requirements through the use of the JAFROC method.

Algorithms↗
PubMed2002

Renal cyst pseudoenhancement: evaluation with an anthropomorphic body CT phantom.

PURPOSE: To determine the effects of cyst diameter and location (intrarenal, exophytic), renal attenuation, section collimation, and computed tomographic (CT) interscanner variability on renal cyst pseudoenhancement in a phantom model. MATERIALS AND METHODS: A customized anthropomorphic phantom was designed to accept 40-, 140-, and 240-HU renal inserts containing intrarenal and exophytic 7-, 10-, and 15-mm cysts. Each phantom and insert were scanned with five different helical CT scanners by using 1.0-1.5-mm, 2.50-3.75-mm, 5.0-mm, 7.0-8.0-mm, and 10.0-mm section collimation. Means and SDs of CT number measurements were obtained for each cyst within each variably "enhanced" renal insert. Mixed-model analysis of variance accommodating heteroscedasticity of data was used to assess the effect of scanner type, section collimation, and cyst diameter on cyst attenuation. RESULTS: Pseudoenhancement (range, 10.3-28.3 HU), observed by using effective section collimation equal to or less than 50% of cyst diameter, occurred in 34 (38%) of 90 intrarenal cyst measurements. Pseudoenhancement was observed with all five CT scanners, though the magnitude of the effect was nonuniform. Significant interactions were noted between renal cyst diameter, background renal attenuation, and CT scanner type in terms of their effects on cyst attenuation. No appreciable pseudoenhancement was observed with exophytic cysts. CONCLUSION: Pseudoenhancement is maximal when small (< or = 1.5-cm) intrarenal cysts are scanned during maximal levels of renal parenchymal enhancement. The magnitude of this effect varies with scanner type but may be large enough to prevent accurate lesion characterization, despite use of a thin-section helical CT data acquisition technique.

Humans↗