Measures of performance in nonlinear estimation tasks: prediction of estimation performance at low signal-to-noise ratio.
Maximum-likelihood (ML) estimation is an established paradigm for the assessment of imaging system performance in nonlinear quantitation tasks. At high signal-to-noise ratio (SNR), ML estimates are asymptotically Gaussian-distributed, unbiased and efficient, thereby attaining the Cramer-Rao bound (CRB). Therefore, at high SNR the CRB is useful as a predictor of the variance of ML estimates and, consequently, as a basis for measures of estimation performance. At low SNR, however, the achievable parameter variances are often substantially larger than the CRB and the estimates are no longer Gaussian-distributed. These departures imply that inference about the estimates that is based on the CRB and the assumption of a normal distribution will not be valid. We have found previously that for some tasks these effects arise at noise levels considered clinically acceptable. We have derived the mathematical relationship between a new measure, chi2(pdf-ML), and the expected probability density of the ML estimates, and have justified the use of chi2(pdf-ML)-isocontours in parameter space to describe the ML estimates. We validated this approach by simulation experiments using spherical objects imaged with a Gaussian point spread function. The parameters, activity concentration and size, were estimated simultaneously by ML, and variances and covariances calculated over 1000 replications per condition from 3D image volumes and from 2D tomographic projections of the same object. At low SNR, where the CRB is no longer achievable, chi2(pdf-ML)-isocontours provide a robust prediction of the distribution of the ML estimates. At high SNR, the chi2(pdf-ML)-isocontours asymptotically approach the analogous chi2(pdf-F)-contours derived from the Fisher information matrix. The chi2(pdf-ML) model appears to be suitable for characterization of the influence of the noise level and characteristics, the task, and the object on the shape of the probability density of the ML estimates at low SNR. Furthermore, it provides unique insights into the causes of the variability of estimation performance.