Exploring subjective vs. objective issues in the validation of computer-based critiquing advice.

Abstract

Evaluation is an important part of the development of computer-based medical expert systems. Such evaluation may be particularly difficult when judging a critiquing system which responds to a proposed management strategy with a discussion of the advisability of that approach. DxCON is an expert system which produces a prose critique discussing the radiologic workup of obstructive jaundice. This paper describes DxCON, and its experimental validation by three independent judges. A central component of the validation involved allowing the judges to react to the system's advice in a quite flexible, unstructured fashion. This project provides a case study of how subjective issues impact both the design and implementation of a validation of a medical expert system whose output is explanatory prose.