Biomedical subjects

Wendy W Chapman

Publications and source records attributed to Wendy W Chapman.

12 recordsLinked to original sources

PubMed2005

Inductive creation of an annotation schema for manually indexing clinical conditions from emergency department reports.

Evaluating automated indexing applications requires comparing automatically indexed terms against manual reference standard annotations. However, there are no standard guidelines for determining which words from a textual document to include in manual annotations, and the vague task can result in substantial variation among manual indexers. We applied grounded theory to emergency department reports to create an annotation schema representing syntactic and semantic variables that could be annotated when indexing clinical conditions. We describe the annotation schema, which includes variables representing medical concepts (e.g., symptom, demographics), linguistic form (e.g., noun, adjective), and modifier types (e.g., anatomic location, severity). We measured the schema's quality and found: (1) the schema was comprehensive enough to be applied to 20 unseen reports without changes to the schema; (2) agreement between author annotators applying the schema was high, with an F measure of 93%; and (3) the authors made complementary errors when applying the schema, demonstrating that the schema incorporates both linguistic and medical expertise.

Abstracting and Indexing↗

PubMed2005

Generating a reliable reference standard set for syndromic case classification.

OBJECTIVE: To generate and measure the reliability for a reference standard set with representative cases from seven broad syndromic case definitions and several narrower syndromic definitions used for biosurveillance. DESIGN: From 527,228 eligible patients between 1990 and 2003, we generated a set of patients potentially positive for seven syndromes by classifying all eligible patients according to their ICD-9 primary discharge diagnoses. We selected a representative subset of the cases for chart review by physicians, who read emergency department reports and assigned values to 14 variables related to the seven syndromes. MEASUREMENTS: (1) Positive predictive value of the ICD-9 diagnoses; (2) prevalence of the syndromic definitions and related variables; (3) agreement between physician raters demonstrated by kappa, kappa corrected for bias and prevalence, and Finn's r; and (4) reliability of the reference standard classifications demonstrated by generalizability coefficients. RESULTS: Positive predictive value for ICD-9 classification ranged from 0.33 for botulinic to 0.86 for gastrointestinal. We generated between 80 and 566 positive cases for six of the seven syndromic definitions. Rash syndrome exhibited low prevalence (34 cases). Agreement between physician raters was high, with kappa > 0.70 for most variables. Ratings showed no bias. Finn's r was >0.70 for all variables. Generalizability coefficients were >0.70 for all variables but three. CONCLUSION: Of the 27 syndromes generated by the 14 variables, 21 showed high enough prevalence, agreement, and reliability to be used as reference standard definitions against which an automated syndromic classifier could be compared. Syndromic definitions that showed poor agreement or low prevalence include febrile botulinic syndrome, febrile and nonfebrile rash syndrome, respiratory syndrome explained by a nonrespiratory or noninfectious diagnosis, and febrile and nonfebrile gastrointestinal syndrome explained by a nongastrointestinal or noninfectious diagnosis.

Bioterrorism↗

PubMed2005

Classification of emergency department chief complaints into 7 syndromes: a retrospective analysis of 527,228 patients.

STUDY OBJECTIVE: Electronic surveillance systems often monitor triage chief complaints in hopes of detecting an outbreak earlier than can be accomplished with traditional reporting methods. We measured the accuracy of a Bayesian chief complaint classifier called CoCo that assigns patients 1 of 7 syndromic categories (respiratory, botulinic, gastrointestinal, neurologic, rash, constitutional, or hemorrhagic) based on free-text triage chief complaints. METHODS: We compared CoCo's classifications with criterion syndromic classification based on International Classification of Diseases, Ninth Revision (ICD-9) discharge diagnoses. We assigned the criterion classification to a patient based on whether the patient's primary diagnosis was a member of a set of ICD-9 codes associated with CoCo's 7 syndromes. We tested CoCo's performance on a set of 527,228 chief complaints from patients registered at the University of Pittsburgh Medical Center emergency department (ED) between 1990 and 2003. We performed a sensitivity analysis by varying the ICD-9 codes in the criterion standard. We also tested CoCo on chief complaints from EDs in a second location (Utah). RESULTS: Approximately 16% (85,569/527,228) of the patients were classified according to the criterion standard into 1 of the 7 syndromes. CoCo's classification performance (number of cases by criterion standard, sensitivity [95% confidence interval (CI)], and specificity [95% CI]) was respiratory (34,916, 63.1 [62.6 to 63.6], 94.3 [94.3 to 94.4]); botulinic (1,961, 30.1 [28.2 to 32.2], 99.3 [99.3 to 99.3]); gastrointestinal (20,431, 69.0 [68.4 to 69.6], 95.6 [95.6 to 95.7]); neurologic (7,393, 67.6 [66.6 to 68.7], 92.7 [92.6 to 92.8]); rash (2,232, 46.8 [44.8 to 48.9], 99.3 [99.3 to 99.3]); constitutional (10,603, 45.8 [44.9 to 46.8], 96.6 [96.6 to 96.7]); and hemorrhagic (8,033, 75.2 [74.3 to 76.2], 98.5 [98.4 to 98.5]). The sensitivity analysis showed that the results were not affected by the choice of ICD-9 codes in the criterion standard. Classification accuracy did not differ on chief complaints from the second location. CONCLUSION: Our results suggest that, for most syndromes, our chief complaint classification system can identify about half of the patients with relevant syndromic presentations, with specificities higher than 90% and positive predictive values ranging from 12% to 44%.

Bayes Theorem↗

PubMed2005

Classifying free-text triage chief complaints into syndromic categories with natural language processing.

OBJECTIVE: Develop and evaluate a natural language processing application for classifying chief complaints into syndromic categories for syndromic surveillance. INTRODUCTION: Much of the input data for artificial intelligence applications in the medical field are free-text patient medical records, including dictated medical reports and triage chief complaints. To be useful for automated systems, the free-text must be translated into encoded form. METHODS: We implemented a biosurveillance detection system from Pennsylvania to monitor the 2002 Winter Olympic Games. Because input data was in free-text format, we used a natural language processing text classifier to automatically classify free-text triage chief complaints into syndromic categories used by the biosurveillance system. The classifier was trained on 4700 chief complaints from Pennsylvania. We evaluated the ability of the classifier to classify free-text chief complaints into syndromic categories with a test set of 800 chief complaints from Utah. RESULTS: The classifier produced the following areas under the ROC curve: Constitutional = 0.95; Gastrointestinal = 0.97; Hemorrhagic = 0.99; Neurological = 0.96; Rash = 1.0; Respiratory = 0.99; Other = 0.96. Using information stored in the system's semantic model, we extracted from the Respiratory classifications lower respiratory complaints and lower respiratory complaints with fever with a precision of 0.97 and 0.96, respectively. CONCLUSION: Results suggest that a trainable natural language processing text classifier can accurately extract data from free-text chief complaints for biosurveillance.

Bayes Theorem↗

PubMed2005

Automating tissue bank annotation from pathology reports - comparison to a gold standard expert annotation set.

Surgical pathology specimens are an important resource for medical research, particularly for cancer research. Although research studies would benefit from information derived from the surgical pathology reports, access to this information is limited by use of unstructured free-text in the reports. We have previously described a pipeline-based system for automated annotation of surgical pathology reports with UMLS concepts, which has been used to code over 450,000 surgical pathology reports at our institution. In addition to coding UMLS terms, it annotates values of several key variables, such as TNM stage and cancer grade. The object of this study was to evaluate the potential and limitations of automated extraction of these variables, by measuring the performance of the system against a true gold standard - manually encoded data entered by expert tissue annotators. We categorized and analyzed errors to determine the potential and limitations of information extraction from pathology reports for the purpose of automated biospecimen annotation.

Abstracting and Indexing↗

PubMed2004

Fever detection from free-text clinical records for biosurveillance.

Automatic detection of cases of febrile illness may have potential for early detection of outbreaks of infectious disease either by identification of anomalous numbers of febrile illness or in concert with other information in diagnosing specific syndromes, such as febrile respiratory syndrome. At most institutions, febrile information is contained only in free-text clinical records. We compared the sensitivity and specificity of three fever detection algorithms for detecting fever from free-text. Keyword CC and CoCo classified patients based on triage chief complaints; Keyword HP classified patients based on dictated emergency department reports. Keyword HP was the most sensitive (sensitivity 0.98, specificity 0.89), and Keyword CC was the most specific (sensitivity 0.61, specificity 1.0). Because chief complaints are available sooner than emergency department reports, we suggest a combined application that classifies patients based on their chief complaint followed by classification based on their emergency department report, once the report becomes available.

Algorithms↗

PubMed2004

Identifying respiratory findings in emergency department reports for biosurveillance using MetaMap.

Clinical conditions described in patients' dictated reports are necessary for automated detection of patients with respiratory illnesses such as inhalational anthrax and pneumonia. We applied MetaMap to emergency department reports to extract a set of 71 clinical conditions relevant to detection of a lower respiratory outbreak. We indexed UMLS terms in emergency department reports with MetaMap, filtered the indexed output with a specialized lexicon of UMLS terms for the domain, and mapped the clinical conditions of interest to concepts in the lexicon. We compared MetaMap's ability to accurately identify the conditions against a physician's manual annotations and evaluated incorrectly indexed features to determine what additional processing is necessary. MetaMap identified the clinical conditions with a recall of 0.72 and a precision of 0.56. Necessary processing beyond MetaMap's indexing includes finding validation, temporal discrimination, anatomic location discrimination, finding-disease discrimination, and contextual inference. Successful identification of clinical conditions in an emergency department report with MetaMap requires processing techniques specific to the clinical question of interest.

Abstracting and Indexing↗

PubMed2004

Implementation and evaluation of a negation tagger in a pipeline-based system for information extract from pathology reports.

We have developed a pipeline-based system for automated annotation of Surgical Pathology Reports with UMLS terms that builds on GATE--an open-source architecture for language engineering. The system includes a module for detecting and annotating negated concepts, which implements the NegEx algorithm--an algorithm originally described for use in discharge summaries and radiology reports. We describe the implementation of the system, and early evaluation of the Negation Tagger. Our results are encouraging. In the key Final Diagnosis section, with almost no modification of the algorithm or phrase lists, the system performs with precision of 0.84 and recall of 0.80 against a gold-standard corpus of negation annotations, created by modified Delphi technique by a panel of pathologists. Further work will focus on refining the Negation Tagger and UMLS Tagger and adding additional processing resources for annotating free-text pathology reports.

Algorithms↗

PubMed2003

Automated syndromic surveillance for the 2002 Winter Olympics.

The 2002 Olympic Winter Games were held in Utah from February 8 to March 16, 2002. Following the terrorist attacks on September 11, 2001, and the anthrax release in October 2001, the need for bioterrorism surveillance during the Games was paramount. A team of informaticists and public health specialists from Utah and Pittsburgh implemented the Real-time Outbreak and Disease Surveillance (RODS) system in Utah for the Games in just seven weeks. The strategies and challenges of implementing such a system in such a short time are discussed. The motivation and cooperation inspired by the 2002 Olympic Winter Games were a powerful driver in overcoming the organizational issues. Over 114,000 acute care encounters were monitored between February 8 and March 31, 2002. No outbreaks of public health significance were detected. The system was implemented successfully and operational for the 2002 Olympic Winter Games and remains operational today.

Algorithms↗

PubMed2003

Electronic interpretation of chest radiograph reports to detect central venous catheters.

OBJECTIVE: To evaluate whether a natural language processing system, SymText, was comparable to human interpretation of chest radiograph reports for identifying the mention of a central venous catheter (CVC), and whether use of SymText could detect patients who had a CVC. DESIGN: To identify patients who had a CVC, we performed two surveys of hospitalized patients. Then, we obtained available reports from 104 patients who had a CVC during one of two cross-sectional surveys (ie, case-patients) and 104 randomly selected patients who did not have a CVC (ie, control-patients). SETTING: A 600-bed public teaching hospital. RESULTS: Chest radiograph reports were available from 124 of the 208 participants. Compared with human interpretation, SymText had a sensitivity of 95.8% and a specificity of 98.7%. The use of SymText to identify case- and control-patients resulted in a sensitivity of 43% and a specificity of 98%. Successful application of SymText varied significantly by venous insertion site (eg, a sensitivity of 78% for subclavian and a sensitivity of 3.7% for femoral). Twenty-six percent of the case-patients had a femoral CVC. CONCLUSIONS: Compared with human interpretation, SymText performed well in interpreting whether a report mentioned a CVC. In patient populations with less frequent CVC placement in femoral veins, the sensitivity for CVC detection likely would be higher. Applying a natural language processing system to chest radiograph reports may be a useful adjunct to other data sources to automate detection of patients who had a CVC.

Catheterization, Central Venous↗

PubMed2002

Rapid deployment of an electronic disease surveillance system in the state of Utah for the 2002 Olympic Winter Games.

The key to minimizing the effects of an intentionally caused disease outbreak is early detection of the attack and rapid identification of the affected individuals. The Bush administration's leadership in advocating for biosurveillance systems capable of monitoring for bioterrorism attacks suggests that we should move quickly to establish a nationwide early warning biosurveillance system as a defense against this threat. The spirit of collaboration and unity inspired by the events of 9-11 and the 2002 Olympic Winter Games in Salt Lake City provided the opportunity to demonstrate how a prototypic biosurveillance system could be rapidly deployed. In seven weeks we were able to implement an automated, real-time disease outbreak detection system in the State of Utah and monitored 80,684 acute care visits occurring during a 28-day period spanning the Olympics. No trends of immediate public health concern were identified.

Bioterrorism↗

PubMed2002

Accuracy of three classifiers of acute gastrointestinal syndrome for syndromic surveillance.

ICD-9-coded emergency department (ED) diagnoses and free-text triage diagnoses are routinely collected data elements that have potential value for public health surveillance and early detection of epidemics. We constructed and measured performance of three classifiers for the detection of cases of acute gastrointestinal syndrome of public health significance: one used ICD-9-coded ED diagnosis as input data; the other two used free-text triage diagnosis. We measured the performance of these classifiers against the expert classification of cases based on review of ED reports. The sensitivity of the ICD-9-code classifier was 0.32, and the specificity was 0.99. The sensitivity of a naïve Bayes classifier using triage diagnoses was 0.63, the specificity was 0.94, and the area under the ROC curve was 0.82. A bigram Bayes classifier had sensitivity 0.38, specificity 0.94, and area under the ROC of 0.69. We conclude that a naive Bayes classifier of free-text triage diagnosis data provides more sensitive and earlier detection of cases of acute gastrointestinal syndrome than either a bigram Bayes classifier or an ICD-9 code classifier. The sensitivity achieved should be sufficient for syndromic surveillance system designed to detect moderate to large epidemics.

Acute Disease↗