Biomedical subjects

Rebecca S Crowley

Publications and source records attributed to Rebecca S Crowley.

12 recordsLinked to original sources

PubMed2007

Evaluation of an intelligent tutoring system in pathology: effects of external representation on performance gains, metacognition, and acceptance.

OBJECTIVE: Determine effects of computer-based tutoring on diagnostic performance gains, meta-cognition, and acceptance using two different problem representations. Describe impact of tutoring on spectrum of diagnostic skills required for task performance. Identify key features of student-tutor interaction contributing to learning gains. DESIGN: Prospective, between-subjects study, controlled for participant level of training. Resident physicians in two academic pathology programs spent four hours using one of two interfaces which differed mainly in external problem representation. The case-focused representation provided an open-learning environment in which students were free to explore evidence-hypothesis relationships within a case, but could not visualize the entire diagnostic space. The knowledge-focused representation provided an interactive representation of the entire diagnostic space, which more tightly constrained student actions. MEASUREMENTS: Metrics included results of pretest, post-test and retention-test for multiple choice and case diagnosis tests, ratios of performance to student reported certainty, results of participant survey, learning curves, and interaction behaviors during tutoring. RESULTS: Students had highly significant learning gains after one tutoring session. Learning was retained at one week. There were no differences between the two interfaces in learning gains on post-test or retention test. Only students in the knowledge-focused interface exhibited significant metacognitive gains from pretest to post-test and pretest to retention test. Students rated the knowledge-focused interface significantly higher than the case-focused interface. CONCLUSIONS: Cognitive tutoring is associated with improved diagnostic performance in a complex medical domain. The effect is retained at one-week post-training. Knowledge-focused external problem representation shows an advantage over case-focused representation for metacognitive effects and user acceptance.

Attitude to Computers↗

PubMed2006

The CAP cancer protocols--a case study of caCORE based data standards implementation to integrate with the Cancer Biomedical Informatics Grid.

BACKGROUND: The Cancer Biomedical Informatics Grid (caBIG) is a network of individuals and institutions, creating a world wide web of cancer research. An important aspect of this informatics effort is the development of consistent practices for data standards development, using a multi-tier approach that facilitates semantic interoperability of systems. The semantic tiers include (1) information models, (2) common data elements, and (3) controlled terminologies and ontologies. The College of American Pathologists (CAP) cancer protocols and checklists are an important reporting standard in pathology, for which no complete electronic data standard is currently available. METHODS: In this manuscript, we provide a case study of Cancer Common Ontologic Representation Environment (caCORE) data standard implementation of the CAP cancer protocols and checklists model--an existing and complex paper based standard. We illustrate the basic principles, goals and methodology for developing caBIG models. RESULTS: Using this example, we describe the process required to develop the model, the technologies and data standards on which the process and models are based, and the results of the modeling effort. We address difficulties we encountered and modifications to caCORE that will address these problems. In addition, we describe four ongoing development projects that will use the emerging CAP data standards to achieve integration of tissue banking and laboratory information systems. CONCLUSION: The CAP cancer checklists can be used as the basis for an electronic data standard in pathology using the caBIG semantic modeling methodology.

Clinical Protocols↗

PubMed2005

An intelligent tutoring system for visual classification problem solving.

OBJECTIVE: This manuscript describes the development of a general intelligent tutoring system for teaching visual classification problem solving. MATERIALS AND METHODS: The approach is informed by cognitive theory, previous empirical work on expertise in diagnostic problem-solving, and our own prior work describing the development of expertise in pathology. The architecture incorporates aspects of cognitive tutoring system and knowledge-based system design within the framework of the unified problem-solving method description language component model. Based on the domain ontology, domain task ontology and case data, the abstract problem-solving methods of the expert model create a dynamic solution graph. Student interaction with the solution graph is filtered through an instructional layer, which is created by a second set of abstract problem-solving methods and pedagogic ontologies, in response to the current state of the student model. RESULTS: In this paper, we outline the empirically derived requirements and design principles, describe the knowledge representation and dynamic solution graph, detail the functioning of the instructional layer, and demonstrate two implemented interfaces to the system. CONCLUSION: Using the general visual classification tutor, we have created SlideTutor, a tutoring system for microscopic diagnosis of inflammatory diseases of skin.

Artificial Intelligence↗

PubMed2005

ReportTutor - an intelligent tutoring system that uses a natural language interface.

ReportTutor is an extension to our work on Intelligent Tutoring Systems for visual diagnosis. ReportTutor combines a virtual microscope and a natural language interface to allow students to visually inspect a virtual slide as they type a diagnostic report on the case. The system monitors both actions in the virtual microscope interface as well as text created by the student in the reporting interface. It provides feedback about the correctness, completeness, and style of the report. ReportTutor uses MMTx with a custom data-source created with the NCI Metathesaurus. A separate ontology of cancer specific concepts is used to structure the domain knowledge needed for evaluation of the student's input including co-reference resolution. As part of the early evaluation of the system, we collected data from 4 pathology residents who typed in their reports without the tutoring aspects of the system, and compared responses to an expert dermatopathologist. We analyzed the resulting reports to (1) identify the error rates and distribution among student reports, (2) determine the performance of the system in identifying features within student reports, and (3) measure the accuracy of the system in distinguishing between correct and incorrect report elements.

Artificial Intelligence↗

PubMed2005

Automating tissue bank annotation from pathology reports - comparison to a gold standard expert annotation set.

Surgical pathology specimens are an important resource for medical research, particularly for cancer research. Although research studies would benefit from information derived from the surgical pathology reports, access to this information is limited by use of unstructured free-text in the reports. We have previously described a pipeline-based system for automated annotation of surgical pathology reports with UMLS concepts, which has been used to code over 450,000 surgical pathology reports at our institution. In addition to coding UMLS terms, it annotates values of several key variables, such as TNM stage and cancer grade. The object of this study was to evaluate the potential and limitations of automated extraction of these variables, by measuring the performance of the system against a true gold standard - manually encoded data entered by expert tissue annotators. We categorized and analyzed errors to determine the potential and limitations of information extraction from pathology reports for the purpose of automated biospecimen annotation.

Abstracting and Indexing↗

PubMed2005

A method for automated detection of usability problems from client user interface events.

Think-aloud usability analysis provides extremely useful data but is very time-consuming and expensive to perform because of the extensive manual video analysis that is required. We describe a simple method for automated detection of usability problems from client user interface events for a developing medical intelligent tutoring system. The method incorporates (1) an agent-based method for communication that funnels all interface events and system responses to a centralized database, (2) a simple schema for representing interface events and higher order subgoals, and (3) an algorithm that reproduces the criteria used for manual coding of usability problems. A correction factor was empirically determining to account for the slower task performance of users when thinking aloud. We tested the validity of the method by simultaneously identifying usability problems using TAU and manually computing them from stored interface event data using the proposed algorithm. All usability problems that did not rely on verbal utterances were detectable with the proposed method.

Algorithms↗

PubMed2004

Implementation and evaluation of a negation tagger in a pipeline-based system for information extract from pathology reports.

We have developed a pipeline-based system for automated annotation of Surgical Pathology Reports with UMLS terms that builds on GATE--an open-source architecture for language engineering. The system includes a module for detecting and annotating negated concepts, which implements the NegEx algorithm--an algorithm originally described for use in discharge summaries and radiology reports. We describe the implementation of the system, and early evaluation of the Negation Tagger. Our results are encouraging. In the key Final Diagnosis section, with almost no modification of the algorithm or phrase lists, the system performs with precision of 0.84 and recall of 0.80 against a gold-standard corpus of negation annotations, created by modified Delphi technique by a panel of pathologists. Further work will focus on refining the Negation Tagger and UMLS Tagger and adding additional processing resources for annotating free-text pathology reports.

Algorithms↗

PubMed2003

Using contextual design to identify potential innovations for problem based learning.

We report on the use of Contextual Design (CD) to develop models of the information management, resource integration, and collaborative processes of medical students in problem-based learning groups. CD is a modified ethnographic technique designed to provide a detailed understanding of the user's needs. Although the technique has been used in non-healthcare related fields, there is limited published data on the application of CD within healthcare settings. In this pilot study, we evaluated the feasibility of the CD methodology for this domain, developed an initial set of CD models, and formulated a series of design ideas based on the data. The study helps to clarify the effectiveness and feasibility of CD as well as the limitations for using this method in health-related domains.

Cooperative Behavior↗

PubMed2003

A general architecture for intelligent tutoring of diagnostic classification problem solving.

We report on a general architecture for creating knowledge-based medical training systems to teach diagnostic classification problem solving. The approach is informed by our previous work describing the development of expertise in classification problem solving in Pathology. The architecture envelops the traditional Intelligent Tutoring System design within the Unified Problem-solving Method description Language (UPML) architecture, supporting component modularity and reuse. Based on the domain ontology, domain task ontology and case data, the abstract problem-solving methods of the expert model create a dynamic solution graph. Student interaction with the solution graph is filtered through an instructional layer, which is created by a second set of abstract problem-solving methods and pedagogic ontologies, in response to the current state of the student model. We outline the advantages and limitations of this general approach, and describe it's implementation in SlideTutor - a developing Intelligent Tutoring System in Dermatopathology.

Artificial Intelligence↗

PubMed2003

The Virtual Slide Set - a curriculum development system for digital microscopy.

We describe the development of a Virtual Slide System for creating and viewing clinico-pathologic cases with embedded interactive digital microscopy. The system supports rich text-to-image annotation, including (1) hotlinks of text descriptions that move the student to the correct part of the slide, and (2) annotations such as arrows and circles that appear on the Virtual Slide on request. The interface can be configured by the student to alter the degree of guidance the system provides. The authoring layer provides a graphical user interface to authors for creating new case sets, cases, questions, and annotated virtual slides, which are saved to a database and automatically added to the Virtual Slide homepage. The system has been used in two pilot studies at the University of Pittsburgh.

Histology↗

PubMed2003

A knowledge-based approach to information extraction from surgical pathology reports.

We describe the development of a prototype system for knowledge-based information extraction from surgical pathology reports. The current system includes abstract problem solving methods and a frame-based knowledge representation of body parts, procedures, diseases, and findings for prostate and breast cases. The system currently extracts the organ, procedure, and diagnoses, and sets an agenda of goals for further processing. A potential advantage of this approach is the ability to increase specificity of information extraction.

Artificial Intelligence↗

PubMedDate not supplied

Development of visual diagnostic expertise in pathology -- an information-processing study.

OBJECTIVE: To identify key features contributing to trainees' development of expertise in microscopic pathology diagnosis, a complex visual task, and to provide new insights to help create computer-based training systems in pathology. DESIGN: Standard methods of information-processing and cognitive science were used to study diagnostic processes (search, perception, reasoning) of 28 novices, intermediates, and experts. Participants examined cases in breast pathology; each case had a previously established gold standard diagnosis. Videotapes correlated the actual visual data examined by participants with their verbal "think-aloud" protocols. MEASUREMENTS: Investigators measured accuracy, difficulty, certainty, protocol process frequencies, error frequencies, and times to key diagnostic events for each case and subject. Analyses of variance, chi-square tests and post-hoc comparisons were performed with subject as the unit of analysis. RESULTS: Level of expertise corresponded with differences in search, perception, and reasoning components of the tasks. Several discrete steps occur on the path to competence, including development of adequate search strategies, rapid and accurate recognition of anatomic location, acquisition of visual data interpretation skills, and transitory reliance on explicit feature identification. CONCLUSION: Results provide the basis for an empirical cognitive model of competence for the complex tasks of microscopic pathology diagnosis. Results will inform the development of computer-based pedagogy tools in this domain

Analysis of Variance↗