PubMed Health⌕ Search

Biomedical subjects

William R Hersh

Publications and source records attributed to William R Hersh.

18 recordsLinked to original sources
PubMed2006

Advancing biomedical image retrieval: development and analysis of a test collection.

OBJECTIVE: Develop and analyze results from an image retrieval test collection. METHODS: After participating research groups obtained and assessed results from their systems in the image retrieval task of Cross-Language Evaluation Forum, we assessed the results for common themes and trends. In addition to overall performance, results were analyzed on the basis of topic categories (those most amenable to visual, textual, or mixed approaches) and run categories (those employing queries entered by automated or manual means as well as those using visual, textual, or mixed indexing and retrieval methods). We also assessed results on the different topics and compared the impact of duplicate relevance judgments. RESULTS: A total of 13 research groups participated. Analysis was limited to the best run submitted by each group in each run category. The best results were obtained by systems that combined visual and textual methods. There was substantial variation in performance across topics. Systems employing textual methods were more resilient to visually oriented topics than those using visual methods were to textually oriented topics. The primary performance measure of mean average precision (MAP) was not necessarily associated with other measures, including those possibly more pertinent to real users, such as precision at 10 or 30 images. CONCLUSIONS: We developed a test collection amenable to assessing visual and textual methods for image retrieval. Future work must focus on how varying topic and run types affect retrieval performance. Users' studies also are necessary to determine the best measures for evaluating the efficacy of image retrieval systems.

Abstracting and Indexing↗
PubMed2006

The TREC 2004 genomics track categorization task: classifying full text biomedical documents.

BACKGROUND: The TREC 2004 Genomics Track focused on applying information retrieval and text mining techniques to improve the use of genomic information in biomedicine. The Genomics Track consisted of two main tasks, ad hoc retrieval and document categorization. In this paper, we describe the categorization task, which focused on the classification of full-text documents, simulating the task of curators of the Mouse Genome Informatics (MGI) system and consisting of three subtasks. One subtask of the categorization task required the triage of articles likely to have experimental evidence warranting the assignment of GO terms, while the other two subtasks were concerned with the assignment of the three top-level GO categories to each paper containing evidence for these categories. RESULTS: The track had 33 participating groups. The mean and maximum utility measure for the triage subtask was 0.3303, with a top score of 0.6512. No system was able to substantially improve results over simply using the MeSH term Mice. Analysis of significant feature overlap between the training and test sets was found to be less than expected. Sample coverage of GO terms assigned to papers in the collection was very sparse. Determining papers containing GO term evidence will likely need to be treated as separate tasks for each concept represented in GO, and therefore require much denser sampling than was available in the data sets. The annotation subtask had a mean F-measure of 0.3824, with a top score of 0.5611. The mean F-measure for the annotation plus evidence codes subtask was 0.3676, with a top score of 0.4224. Gene name recognition was found to be of benefit for this task. CONCLUSION: Automated classification of documents for GO annotation is a challenging task, as was the automated extraction of GO code hierarchies and evidence codes. However, automating these tasks would provide substantial benefit to biomedical curation, and therefore work in this area must continue. Additional experience will allow comparison and further analysis about which algorithmic features are most useful in biomedical document classification, and better understanding of the task characteristics that make automated classification feasible and useful for biomedical document curation. The TREC Genomics Track will be continuing in 2005 focusing on a wider range of triage tasks and improving results from 2004.

Journal Article↗
PubMed2006

Enhancing access to the Bibliome: the TREC 2004 Genomics Track.

BACKGROUND: The goal of the TREC Genomics Track is to improve information retrieval in the area of genomics by creating test collections that will allow researchers to improve and better understand failures of their systems. The 2004 track included an ad hoc retrieval task, simulating use of a search engine to obtain documents about biomedical topics. This paper describes the Genomics Track of the Text Retrieval Conference (TREC) 2004, a forum for evaluation of IR research systems, where retrieval in the genomics domain has recently begun to be assessed. RESULTS: A total of 27 research groups submitted 47 different runs. The most effective runs, as measured by the primary evaluation measure of mean average precision (MAP), used a combination of domain-specific and general techniques. The best MAP obtained by any run was 0.4075. Techniques that expanded queries with gene name lists as well as words from related articles had the best efficacy. However, many runs performed more poorly than a simple baseline run, indicating that careful selection of system features is essential. CONCLUSION: Various approaches to ad hoc retrieval provide a diversity of efficacy. The TREC Genomics Track and its test collection resources provide tools that allow improvement in information retrieval systems.

Journal Article↗
PubMed2006

Diagnosis, access and outcomes: Update of a systematic review of telemedicine services.

Telemedicine services are being increasingly used. Although insurers and other payers are covering some services in the USA, the rationale for these coverage decisions is not always evidence-based. We reviewed the literature for telemedicine services that substitute for face-to-face medical diagnosis and treatment. We focused on three types of telemedicine services: store-and-forward, home-based and office/hospital-based services. Studies were included if they were relevant to at least one of the three study areas, addressed at least one key question and contained reported results. We excluded articles that did not study a service requiring face-to-face encounters (i.e. teleradiology was excluded). Our search initially identified 4083 citations. After review, 597 were judged to be potentially relevant at the title/abstract level. Following a full-text review, 106 studies were included. Store-and-forward services have been studied in many specialties, the most common being dermatology, wound care and ophthalmology. The evidence for their efficacy is mixed. Several limited studies showed the benefits of home-based telemedicine interventions in chronic diseases. Studies of office/hospital-based telemedicine suggest that telemedicine is most effective for verbal interactions, e.g. videoconferencing for diagnosis and treatment in specialties like neurology and psychiatry. There are still significant gaps in the evidence base between where telemedicine is used and where its use is supported by high-quality evidence. Further well-designed research is necessary to understand how best to deploy telemedicine services in health care.

Delivery of Health Care↗
PubMed2005

Using citation data to improve retrieval from MEDLINE.

OBJECTIVE: To determine whether algorithms developed for the World Wide Web can be applied to the biomedical literature in order to identify articles that are important as well as relevant. DESIGN AND MEASUREMENTS A direct comparison of eight algorithms: simple PubMed queries, clinical queries (sensitive and specific versions), vector cosine comparison, citation count, journal impact factor, PageRank, and machine learning based on polynomial support vector machines. The objective was to prioritize important articles, defined as being included in a pre-existing bibliography of important literature in surgical oncology. RESULTS Citation-based algorithms were more effective than noncitation-based algorithms at identifying important articles. The most effective strategies were simple citation count and PageRank, which on average identified over six important articles in the first 100 results compared to 0.85 for the best noncitation-based algorithm (p < 0.001). The authors saw similar differences between citation-based and noncitation-based algorithms at 10, 20, 50, 200, 500, and 1,000 results (p < 0.001). Citation lag affects performance of PageRank more than simple citation count. However, in spite of citation lag, citation-based algorithms remain more effective than noncitation-based algorithms. CONCLUSION Algorithms that have proved successful on the World Wide Web can be applied to biomedical information retrieval. Citation-based algorithms can help identify important articles within large sets of relevant results. Further studies are needed to determine whether citation-based algorithms can effectively meet actual user information needs.

Algorithms↗
PubMed2005

A survey of current work in biomedical text mining.

The volume of published biomedical research, and therefore the underlying biomedical knowledge base, is expanding at an increasing rate. Among the tools that can aid researchers in coping with this information overload are text mining and knowledge extraction. Significant progress has been made in applying text mining to named entity recognition, text classification, terminology extraction, relationship extraction and hypothesis generation. Several research groups are constructing integrated flexible text-mining systems intended for multiple uses. The major challenge of biomedical text mining over the next 5-10 years is to make these systems useful to biomedical researchers. This will require enhanced access to full text, better understanding of the feature space of biomedical literature, better methods for measuring the usefulness of systems to users, and continued cooperation with the biomedical research community to ensure that their needs are addressed.

Abstracting and Indexing↗
PubMed2004

A categorization and analysis of the criticisms of Evidence-Based Medicine.

The major criticisms and limitations of Evidence-Based Medicine (EBM) appearing in the literature over the past decade can be summarized and categorized into five recurring themes. The themes include: reliance on empiricism, narrow definition of evidence, lack of evidence of efficacy, limited usefulness for individual patients, and threats to the autonomy of the doctor/patient relationship. Analysis of EBM according to these themes leads to the conclusion that EBM can be a useful tool, but has severe drawbacks when used in isolation in the practice of individual patient care. Modern medicine must strive to balance an extremely complex set of priorities. To be an effective aid in achieving this balance, the theory and practice of EBM must expand to include new methods of study design and integration, and must adapt to the needs of both patients and the health care system in order to provide patients with the best care at the lowest cost.

Attitude to Health↗
PubMed2003

Computerized physician order entry in U.S. hospitals: results of a 2002 survey.

OBJECTIVE: To determine the availability of inpatient computerized physician order entry in U.S. hospitals and the degree to which physicians are using it. DESIGN: Combined mail and telephone survey of 964 randomly selected hospitals, contrasting 2002 data and results of a survey conducted in 1997. MEASUREMENTS AVAILABILITY: computerized order entry has been installed and is available for use by physicians; inducement: the degree to which use of computers to enter orders is required of physicians; participation: the proportion of physicians at an institution who enter orders by computer; and saturation: the proportion of total orders at an institution entered by a physician using a computer. RESULTS: The response rate was 65%. Computerized order entry was not available to physicians at 524 (83.7%) of 626 hospitals responding, whereas 60 (9.6%) reported complete availability and 41 (6.5%) reported partial availability. Of 91 hospitals providing data about inducement/requirement to use the system, it was optional at 31 (34.1%), encouraged at 18 (19.8%), and required at 42 (46.2%). At 36 hospitals (45.6%), more than 90% of physicians on staff use the system, whereas six (7.6%) reported 51-90% participation and 37 (46.8%) reported participation by fewer than half of physicians. Saturation was bimodal, with 25 (35%) hospitals reporting that more than 90% of all orders are entered by physicians using a computer and 20 (28.2%) reporting that less than 10% of all orders are entered this way. CONCLUSION: Despite increasing consensus about the desirability of computerized physician order entry (CPOE) use, these data indicate that only 9.6% of U.S. hospitals presently have CPOE completely available. In those hospitals that have CPOE, its use is frequently required. In approximately half of those hospitals, more than 90% of physicians use CPOE; in one-third of them, more than 90% of orders are entered via CPOE.

Data Collection↗
PubMed2003

A pilot study of contextual UMLS indexing to improve the precision of concept-based representation in XML-structured clinical radiology reports.

OBJECTIVE: Despite the advantages of structured data entry, much of the patient record is still stored as unstructured or semistructured narrative text. The issue of representing clinical document content remains problematic. The authors' prior work using an automated UMLS document indexing system has been encouraging but has been affected by the generally low indexing precision of such systems. In an effort to improve precision, the authors have developed a context-sensitive document indexing model to calculate the optimal subset of UMLS source vocabularies used to index each document section. This pilot study was performed to evaluate the utility of this indexing approach on a set of clinical radiology reports. DESIGN: A set of clinical radiology reports that had been indexed manually using UMLS concept descriptors was indexed automatically by the SAPHIRE indexing engine. Using the data generated by this process the authors developed a system that simulated indexing, at the document section level, of the same document set using many permutations of a subset of the UMLS constituent vocabularies. MEASUREMENTS: The precision and recall scores generated by simulated indexing for each permutation of two or three UMLS constituent vocabularies were determined. RESULTS: While there was considerable variation in precision and recall values across the different subtypes of radiology reports, the overall effect of this indexing strategy using the best combination of two or three UMLS constituent vocabularies was an improvement in precision without significant impact of recall. CONCLUSION: In this pilot study a contextual indexing strategy improved overall precision in a set of clinical radiology reports.

Abstracting and Indexing↗
PubMed2002

SmartQuery: context-sensitive links to medical knowledge sources from the electronic patient record.

Physicians have many unmet information needs that arise in the course of patient care. Many clinical questions could potentially be answered by streamlined access to medical literature, textbooks, and clinical guidelines in the context of the electronic medical record. We designed and implemented SmartQuery, a prototype application to provide context-sensitive links from an electronic patient record to relevant medical knowledge sources, then performed a preliminary user evaluation. Our results suggest that such an application may be clinically useful, and provide some insight into problems and priorities for future development.

Expert Systems↗
PubMed2002

Distributed medical informatics education using internet2.

The curricula of most medical informatics training programs are incomplete. We used Internet2-based videoconferencing to expand the educational opportunities of medical informatics students at Oregon Health & Science University and the University of Pittsburgh. Students and faculty in both programs shared extra-curricular research conferences and journal club meetings. A course in Information Retrieval was made available to students in both programs. The conferences, meetings and class were well accepted by participants. A few problems were experienced with the technology, some of which were resolved, and some non-technical challenges to distributing academic conferences, meetings and coursework were also uncovered. We plan to continue our efforts with expanded course and extra-curricular offerings and a more comprehensive evaluation strategy.

Education, Distance↗
PubMedDate not supplied

Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions.

OBJECTIVES: This study sought to assess the ability of medical and nurse practitioner students to use MEDLINE to obtain evidence for answering clinical questions and to identify factors associated with the successful answering of questions. METHODS: A convenience sample of medical and nurse practitioner students was recruited. After completing instruments measuring demographic variables, computer and searching attitudes and experience, and cognitive traits, the subjects were given a brief orientation to MEDLINE searching and the techniques of evidence-based medicine. The subjects were then given 5 questions (from a pool of 20) to answer in two sessions using the Ovid MEDLINE system and the Oregon Health & Science University library collection. Each question was answered using three possible responses that reflected the quality of the evidence. All actions capable of being logged by the Ovid system were captured. Statistical analysis was performed using a model based on generalized estimating equations. The relevance-based measures of recall and precision were measured by defining end queries and having relevance judgments made by physicians who were not associated with the study. RESULTS: Forty-five medical and 21 nurse practitioner students provided usable answers to 324 questions. The rate of correctness increased from 32.3 to 51.6 percent for medical students and from 31.7 to 34.7 percent for nurse practitioner students. Ability to answer questions correctly was most strongly associated with correctness of the answer before searching, user experience with MEDLINE features, the evidence-based medicine question type, and the spatial visualization score. The spatial visualization score showed multi-colinearity with student type (medical vs. nurse practitioner). Medical and nurse practitioner students obtained comparable recall and precision, neither of which was associated with correctness of the answer. CONCLUSIONS: Medical and nurse practitioner students in this study were at best moderately successful at answering clinical questions correctly with the assistance of literature searching. The results confirm the importance of evaluating both search ability and the ability to use the resulting information to accomplish a clinical task.

Adult↗
PubMedDate not supplied

Does national regulatory mandate of provider order entry portend greater benefit than risk for health care delivery? The 2001 ACMI debate. The American College of Medical Informatics.

The 2001 debate of the American College of Medical Informatics focused on the proposition that national regulatory mandate of computer-based provider order entry (CPOE), to take effect by the end of 2005, portends greater benefit than risk for health care delivery. Both sides accepted that provider order entry offers potential benefit. Those supporting the proposition emphasized public safety, noting that payers have little economic incentive to pay for quality and that a mandate would force vendors to improve the usability and value of their systems. They argued that the mandate would align the economic incentives to finally allow CPOE to be widely adopted. Those opposing the proposition emphasized the risks resulting from a mandate, including the direct implementation costs, the logistic issues of implementation, and the cost of failed implementations. They also noted the potential for errors introduced by the systems themselves and the fact that the safety and utility of commercially available CPOE products have yet to be proved.

Delivery of Health Care↗