Evaluating a perimetric expert system: experience with Octosmart.

Abstract

When evaluating expert systems to be used in clinical perimetry, various aspects of their performance as compared with that of human interpreters must be considered. In this investigation, the results produced by the new Octosmart diagnostic program have been compared with the performance of three interpreters with various amounts of experience in visual field analysis. The evaluations were based on 27 visual fields with glaucomatous damage, which had been examined with the Octopus program G1. It is shown that in borderline cases (i.e., neither clearly normal nor clearly pathological) where strict statistical criteria must be employed in order to distinguish between possible pathology and artifacts, the "personal styles" of human interpreters, more than standardized decision criteria, implicitly guide the decision process, resulting in unpredictable, non-standardized interindividual differences. A standardized expert system, based on constant, explicit, and logical criteria is therefore considered to be superior to unaided human interpretation. It is pointed out that the influence of the implicit decision criteria of human interpreters must be controlled carefully if expert systems are to be evaluated with reference to human interpreters.