Biomedical subjects

P J Haug

Publications and source records attributed to P J Haug.

At least 19 recordsLinked to original sources

PubMed2003

Building a comprehensive clinical information system from components. The approach at Intermountain Health Care.

OBJECTIVES: To discuss the advantages and disadvantages of an interfaced approach to clinical information systems architecture. METHODS: After many years of internally building almost all components of a hospital clinical information system (HELP) at Intermountain Health Care, we changed our architectural approach as we chose to encompass ambulatory as well as acute care. We now seek to interface applications from a variety of sources (including some that we build ourselves) to a clinical data repository that contains a longitudinal electronic patient record. RESULTS: We have a total of 820 instances of interfaces to 51 different applications. We process nearly 2 million transactions per day via our interface engine and feel that the reliability of the approach is acceptable. Interface costs constitute about four percent of our total information systems budget. The clinical database currently contains records for 1.45 m patients and the response time for a query is 0.19 sec. DISCUSSION: Based upon our experience with both integrated (monolithic) and interfaced approaches, we conclude that for those with the expertise and resources to do so, the interfaced approach offers an attractive alternative to systems provided by a single vendor. We expect the advantages of this approach to increase as the costs of interfaces are reduced in the future as standards for vocabulary and messaging become increasingly mature and functional.

Computer Systems↗

PubMed2001

A comparison of classification algorithms to automatically identify chest X-ray reports that support pneumonia.

We compared the performance of expert-crafted rules, a Bayesian network, and a decision tree at automatically identifying chest X-ray reports that support acute bacterial pneumonia. We randomly selected 292 chest X-ray reports, 75 (25%) of which were from patients with a hospital discharge diagnosis of bacterial pneumonia. The reports were encoded by our natural language processor and then manually corrected for mistakes. The encoded observations were analyzed by three expert systems to determine whether the reports supported pneumonia. The reference standard for radiologic support of pneumonia was the majority vote of three physicians. We compared (a) the performance of the expert systems against each other and (b) the performance of the expert systems against that of four physicians who were not part of the gold standard. Output from the expert systems and the physicians was transformed so that comparisons could be made with both binary and probabilistic output. Metrics of comparison for binary output were sensitivity (sens), precision (prec), and specificity (spec). The metric of comparison for probabilistic output was the area under the receiver operator characteristic (ROC) curve. We used McNemar's test to determine statistical significance for binary output and univariate z-tests for probabilistic output. Measures of performance of the expert systems for binary (probabilistic) output were as follows: Rules--sens, 0.92; prec, 0.80; spec, 0.86 (Az, 0.960); Bayesian network--sens, 0.90; prec, 0.72; spec, 0.78 (Az, 0.945); decision tree--sens, 0.86; prec, 0.85; spec, 0.91 (Az, 0.940). Comparisons of the expert systems against each other using binary output showed a significant difference between the rules and the Bayesian network and between the decision tree and the Bayesian network. Comparisons of expert systems using probabilistic output showed no significant differences. Comparisons of binary output against physicians showed differences between the Bayesian network and two physicians. Comparisons of probabilistic output against physicians showed a difference between the decision tree and one physician. The expert systems performed similarly for the probabilistic output but differed in measures of sensitivity, precision, and specificity produced by the binary output. All three expert systems performed similarly to physicians.

Acute Disease↗

PubMed2001

Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia.

RATIONALE AND OBJECTIVES: The purpose of this study was to statistically identify some characteristics of unambiguous (ie, clear) chest radiography reports in the context of acute bacterial pneumonia. MATERIALS AND METHODS: Seven physicians individually read 292 chest radiography reports to determine if they contained radiologic evidence of pneumonia. Unambiguous reports were defined as those that physicians unanimously classified as supporting or not supporting the diagnosis of pneumonia. Ambiguous reports were assigned degrees of ambiguity on the basis of how much disagreement they caused among the physicians. Characteristics of unambiguous reports as described in the literature were manually quantified and assigned to every report. To identify characteristics that statistically distinguished unambiguous from ambiguous reports, the authors performed an ordinal logistic regression analysis for which the dependent variable was the number of dissenting votes the report received and the independent variables were the quantified characteristics of the report. RESULTS: Six independent variables were statistically significantly associated with unambiguous reports (P < .05). Three were positively associated: an interpretation of whether findings supported the diagnosis of pneumonia in reports with pneumonia-related observations, short sentences, and redundancy of pneumonia-related observations. Three were negatively associated: high use of uncertainty modifiers for pneumonia-related observations, use of only descriptive terms to describe pneumonia-related observations, and insufficient amount of pneumonia-related information. CONCLUSION: The most influential characteristic of an unambiguous chest radiography report was an interpretation of whether the radiograph supported the diagnosis of pneumonia when findings could be indicative.

Diagnosis, Differential↗

PubMed2001

A comprehensive set of coded chief complaints for the emergency department.

OBJECTIVE: To develop a generally applicable set of coded chief complaints for the computerized patient records of emergency departments (EDs). METHODS: At an urban teaching ED the chief complaints of more than 50,000 patients were analyzed retrospectively during a 29-month period (June 1995-October 1997). Applying continuous quality improvement methods, a multidisciplinary team examined the current process documenting the patient's chief complaint. During two prospective periods (November 1997-December 1998; January 1999-June 1999), more than 34,000 chief complaints were analyzed. To reduce free-text charting practices, a variety of interventions on individual and team level were applied. Quantitative analysis was performed with statistical process control charts, and a qualitative evaluation was performed with a questionnaire. RESULTS: The charting of chief complaint in free-text format decreased from 23% to 1%. The range among individual ED staff members narrowed from 45% to 9%. During the refinement of the set of coded chief complaints, six infrequently charted items were removed. Five new chief complaints identified by analysis of free-text entries during the second study period were added. The current set of chief complaints consists of 54 codable and the three original free-text items. The ED staff members perceived all the interventions beneficial. A poster displaying all available terms as a visual aid, however, had the largest impact on charting the patient's chief complaint in coded format. CONCLUSIONS: Applying continuous quality improvement methods, the authors created a clinically developed and applicable set of codable chief complaints that can be easily integrated into a computerized patient record of an ED.

Emergency Service, Hospital↗

PubMed2001

Automatic identification of patients eligible for a pneumonia guideline: comparing the diagnostic accuracy of two decision support models.

BACKGROUND: In busy clinical settings, physicians often do not have enough time to identify patients for specific therapeutic guidelines. As a solution, decision support systems could automatically identify eligible patients and trigger computerized guidelines for specific diseases. Applying this idea to community-acquired pneumonia (CAP), we developed a Bayesian network (BN) and an artificial neural network (ANN) for identifying patients who have CAP and are eligible for a pneumonia guideline. OBJECTIVE: The aim of this study was to determine whether the diagnostic accuracy of these two decision support models differs in terms of identifying CAP patients. METHODS: We trained and tested the networks with a data set of 32,662 adult patients. For each network, we (1) calculated the specificity, the positive predictive value (PPV), and the negative predictive value (NPV) at a sensitivity of 95%, and (2) determined the area under the receiver operating characteristic curve (AUC) as a measure of overall accuracy. We tested for statistical difference between the AUCs using the correlated area z statistic. RESULTS: At a sensitivity of 95%, the respective values for specificity, PPV, and NPV were: 92.3%, 15.1%, and 99.9% for the BN, and 94.0%, 18.6%, and 99.9% for the ANN. The BN had an AUC of 0.9795 (95% CI: 0.9736, 0.9843), and the ANN had an AUC of 0.9855 (95% CI: 0.9805, 0.9894). The difference between the AUCs was statistically significant (p=0.0044). CONCLUSIONS: The networks achieved high overall accuracies on the testing data set. Because the difference in accuracies is statistically significant but not clinically significant, both networks are equally suited to drive a guideline.

Area Under Curve↗

PubMed2001

Combining decision support methodologies to diagnose pneumonia.

OBJECTIVE: To evaluate the performance of a computerized decision support system that combines two different decision support methodologies (a Bayesian network and a natural language understanding system) for the diagnosis of patients with pneumonia. DESIGN: Evaluation study using data from a prospective, clinical study. PATIENTS: All patients 18 years and older who presented to the emergency department of a tertiary care setting and whose chest x-ray report was available during the encounter. METHODS: The computerized decision support system calculated a probability of pneumonia using information provided by the two systems. Outcome measures were the area under the receiver operating characteristic curve, sensitivity, specificity, predictive values, likelihood ratios, and test effectiveness. RESULTS: During the 3-month study period there were 742 patients (45 with pneumonia). The area under the receiver operating characteristic curve was 0.881 (95% CI: 0.822, 0.925) for the Bayesian network alone and 0.916 (95% CI: 0.869, 0.949) for the Bayesian network combined with the natural language understanding system (p=0.01). CONCLUSION: Combining decision support methodologies that process information stored in different data formats can increase the performance of a computerized decision support system.

Adult↗

PubMed2000

Automatic identification of patients eligible for a pneumonia guideline.

OBJECTIVE: To assess the ability of an integrated, real-time diagnostic system (Bayesian network) to identify patients with community-acquired pneumonia who are eligible for a computerized pneumonia guideline without requiring clinicians to enter additional data. DESIGN: Prospective validation study. PATIENTS: All patients 18 years and older who presented to the emergency department of a tertiary care hospital. METHODS: The diagnostic system computed a probability of pneumonia for every patient. The final diagnosis was established using ICD-9 discharge diagnoses. Outcome measures were sensitivity, specificity, predictive values, likelihood ratios, area under the receiver operating characteristic curve, and test effectiveness. RESULTS: During the 9-week study period there were 4,361 patients (112 pneumonia patients). The area under the receiver operating characteristic curve was 0.930 (CI: 0.907, 0.948). At a fixed sensitivity of 95%, the specificity was 68.5%, the positive predictive value 7.3%, the negative predictive value 99.8%, the positive likelihood ratio 3.0, the negative likelihood ratio 0.08, and the test effectiveness 2.05. CONCLUSION: The diagnostic system was able to detect patients who are eligible for a pneumonia guideline. The detection of eligible patients can be applied to automatically initiate and evaluate computerized guidelines.

Adult↗

PubMed2000

Contribution of a speech recognition system to a computerized pneumonia guideline in the emergency department.

OBJECTIVE: Evaluate the effect of a radiology speech recognition system on a real-time computerized guideline in the emergency department. METHODS: We collected all chest x-ray reports (n = 727) generated for patients in the emergency department during a six-week period. We divided the concurrently generated reports into those generated with speech recognition and those generated by traditional dictation. We compared the two sets of reports for availability during the patient's emergency department encounter and for readability. RESULTS: Reports generated by speech recognition were available seven times more often during the patients' encounters than reports generated by traditional dictation. Using speech recognition reduced the turnover time of reports from 12 hours 33 minutes to 2 hours 13 minutes. Readability scores were identical for both kinds of reports. CONCLUSION: Using speech recognition to generate chest x-ray reports reduces turnover time so reports are available while patients are in the emergency department.

Decision Making, Computer-Assisted↗

PubMed2000

Using medical language processing to support real-time evaluation of pneumonia guidelines.

OBJECTIVE: To evaluate if a medical language processing (MLP) system is able to support real-time computerization of community-acquired pneumonia (CAP) guidelines. METHODS: Prospective validation study in the emergency department of a tertiary care facility. All the chest x-ray reports available in real-time for an admission decision during a five-week period were included. The MLP system was compared to a physician for the automatic selection of eligible patients and on the extraction of radiographic findings required by five different CAP guidelines. The gold standard comprised of three independent physicians and reliability measures were calculated. The outcome measures were the area under the receiver operated characteristic curve (AUC) for selecting eligible patients, sensitivity, positive predictive value (PPV), and specificity for the extraction of radiographic findings. RESULTS: During the five-week period, 243 reports were available in real-time. The AUCs on selecting eligible CAP patients were 89.7% (CI: 84.2%, 93.7%) for the MLP system, and 93.3% (CI: 83.9%, 97.8%) for the physician. The average sensitivity, PPV, and specificity for radiographic findings that assessed localization and extension of CAP were respectively: 94%, 87%, 96% (physician); and 34%, 90%, 95% (MLP system). Both, the MLP system and the physician had average sensitivity, PPV, and specificity of 97%, 97%, and 99%, respectively, when localization was not an issue. Reliability measures for the gold standard were above 70%. CONCLUSION: The MLP system was able to support real-time computerization of guidelines by selecting eligible patients and extracting radiographic findings that do not assess localization and extension of CAP.

Community-Acquired Infections↗

PubMed1999

Automatic identification of pneumonia related concepts on chest x-ray reports.

A medical language processing system called SymText, two other automated methods, and a lay person were compared against an internal medicine resident for their ability to identify pneumonia related concepts on chest x-ray reports. Sensitivity (recall), specificity, and positive predictive value (precision) are reported with respect to an independent panel of physicians. Overall the performance of SymText was similar to the physician and superior to the other methods. The automatic encoding of pneumonia concepts will support clinical research, decision making, computerized clinical protocols, and quality assurance in a radiology department.

Bayes Theorem↗

PubMed1999

An integrated decision support system for diagnosing and managing patients with community-acquired pneumonia.

Decision support systems that integrate guidelines have become popular applications to reduce variation and deliver cost-effective care. However, adverse characteristics of decision support systems, such as additional and time-consuming data entry or manually identifying eligible patients, result in a "behavioral bottleneck" that prevents decision support systems to become part of the clinical routine. This paper describes the design and the implementation of an integrated decision support system that explores a novel approach for bypassing the behavioral bottleneck. The real-time decision support system does not require health care providers to enter additional data and consists of a diagnostic and a management component.

Bayes Theorem↗

PubMed1999

Comparing expert systems for identifying chest x-ray reports that support pneumonia.

We compare the performance of four computerized methods in identifying chest x-ray reports that support acute bacterial pneumonia. Two of the computerized techniques are constructed from expert knowledge, and two learn rules and structure from data. The two machine learning systems perform as well as the expert constructed systems. All of the computerized techniques perform better than a baseline keyword search and a lay person, and perform as well as a physician. We conclude that machine learning can be used to identify chest x-ray reports that support pneumonia.

Algorithms↗

PubMed1998

Evaluation of a semantic data model for chest radiology: application of a new methodology.

An essential step toward the effective processing of the medical language is the development of representational models that formalize the language semantics. These models, also known as semantic data models, help to unlock the meaning of descriptive expressions, making them accessible to computer systems. The present study tries to determine the quality of a semantic data model created to encode chest radiology findings. The evaluation methodology relied on the ability of physicians to extract information from textual and encoded representations of chest X-ray reports, whilst answering questions associated with each report. The evaluation demonstrated that the encoded reports seemed to have the same information content of the original textual reports. The methodology generated useful data regarding the quality of the data model, demonstrating that certain segments were creating ambiguous representations and that some details were not being represented.

Adult↗

PubMed1998

Bayesian modeling for linking causally related observations in chest X-ray reports.

Our natural language understanding system outputs a list of diseases, findings, and appliances found in a chest x-ray report. The system described in this paper links those diseases and findings that are causally related. Using Bayesian networks to model the conceptual and diagnostic information found in a chest x-ray we are able to infer more specific information about the findings that are linked to diseases.

Algorithms↗

PubMed1998

Diagnosing community-acquired pneumonia with a Bayesian network.

We present the development and the evaluation of a Bayesian network for the diagnosis of community-acquired pneumonia. The Bayesian network is intended to be part of a larger decision support system which assists emergency room physicians in the management of pneumonia patients. Minimal data entry from the nurse or the physician, timely availability of clinical parameters, and high accuracy were requirements we tried to meet. Data from more than 32,000 emergency room patients over a period of 2 years (June 1995-June 1997) were extracted from the clinical information system to train and test the Bayesian network. The network performed well in discriminating patients with pneumonia from patients with other diseases. The Bayesian network achieved a sensitivity of 95%, a specificity of 96.5%, an area under the receiver operating characteristic of 0.98, and a predictive value positive of 26.8%. Our feasibility study demonstrates that the proposed Bayesian network is an appropriate method to detect pneumonia patients with high accuracy. The study suggests that the proposed Bayesian network may represent a successful component within a larger decision support system for the management of community-acquired pneumonia.

Adult↗

PubMed1998

Automatic extraction of PIOPED interpretations from ventilation/perfusion lung scan reports.

Free-text documents are the main type of data produced by a radiology department in a hospital information system. While this type of data is readily accessible for clinical data review it can not be accessed by other applications to perform medical decision support, quality assurance, and outcome studies. In an attempt to solve this problem, natural language processing systems have been developed and tested against chest x-rays reports to extract relevant clinical information and make it accessible to other computer applications. We have used a natural language processing tool called SymText to extract relevant clinical information from a different type of radiology report, the Ventilation/Perfusion lung scan report. Results of this effort can be analyzed in terms of precision and recall. The overall precision was 0.88 and recall was 0.92. In addition, the natural language processing system functions differently in reports with and without an impression section. If this type of information can be successfully extracted from radiology reports, one can develop quality monitors for the diagnostic performance of the radiologist by correlating the impressions with gold standard data present in a hospital information system. Avoiding the manual effort previously necessary to create quality assurance data, can lead to a higher frequency of quality review in a radiology department.

Bayes Theorem↗

PubMed1997

Developing a radiology data base for quality assurance.

Radiology Information Systems (RIS) are designed to capture and manage the data associated with ordering, executing, reporting, and billing x-ray procedures. The HELP Hospital Information System contains a radiology subsystem that supports these functions. In an effort to enhance quality assurance initiatives, we have created a supplemental data base. This data base contains not only the data traditionally generated by RISs but also data from the hospital system that is relevant to quality assurance. One of the goals associated with this data base is to use techniques from the discipline of Continuous Quality Improvement (CQI) in the radiology department. A focus of our initial efforts has been the time necessary to provide x-ray reports to ordering physicians once the imaging examination has been performed. Efforts to manage the portion of this time interval caused by transcription have resulted in a substantial decrease in the time required for this function. A second goal of this project is to evaluate the quality of x-ray ordering. This objective requires a computerized record of the outcome of the x-ray procedure. Initial analysis of data derived from this data base indicates significant differences in the ordering behavior for computed tomography (CT) examinations among a test group of physicians. A third goal is to do quality assurance on x-ray reports. Experience with pilot systems has shown promising results using a mathematical model of report quality. We hope to leverage these techniques and this quality assurance data base to define a COI process for medical reports in general and for x-ray reports in particular.

Database Management Systems↗

PubMed1997

Inducing practice guidelines from a hospital database.

Improving health care quality requires the elimination of unnecessary variation in the care process. Decision support applications already exist that can foster adherence to standards. The challenge resides in developing standards consistent with good medical practice. In this paper we present our efforts in determining where sufficient clinical data are captured electronically to automatically define a care process, and what analyses can be done to identify additional data that would allow a care process to be defined. Data routinely collected by a hospital information system have been examined. The analysis tools utilized include logistic regression, a neural network, a Bayesian network, and a rule induction program.

Bayes Theorem↗