Response to Corrao et al.: Improving efficacy of PubMed clinical queries for retrieving scientifically strong studies on treatment.
Explore the source record for details and available documents.
Biomedical subjects
Publications and source records attributed to R Brian Haynes.
Explore the source record for details and available documents.
Knowledge translation describes any activity or process that facilitates the transfer of high-quality evidence from research into effective changes in health policy, clinical practice, or products. This increasingly important discipline attempts to conceptually combine elements of research, education, quality improvement, and electronic systems development to create a seamless linkage between interventions that improve patient care and their routine implementation in daily clinical practice. We outline the gap between research and practice and present a case study of an emergency medicine example of validated evidence that has failed to achieve widespread implementation. The authors describe a model of organization of evidence and its relationship with the process that links research from the scientific endeavor to changes in practice that affect patient outcomes. Obstacles to evidence uptake are explored, as well as the limitations of current educational strategies. Innovative strategies in realms such as computerized decision support systems designed to enhance evidence uptake are also described. The potential interface between knowledge translation and continuous quality improvement, as well as the role for bedside tools, is also presented. Research in knowledge translation includes studies that attempt to quantify and understand the discrepancies between what is known and what is done, as well as those that examine the impact and acceptability of interventions designed to narrow or close these gaps. Sentinel examples in this line of research conducted in the emergency department setting are described.
Explore the source record for details and available documents.
BACKGROUND: Physicians have difficulty keeping up with new evidence from medical research. METHODS: We developed the McMaster Premium LiteratUre Service (PLUS), an internet-based addition to an existing digital library, which delivered quality- and relevance-rated medical literature to physicians, matched to their clinical disciplines. We evaluated PLUS in a cluster-randomized trial of 203 participating physicians in Northern Ontario, comparing a Full-Serve version (that included alerts to new articles and a cumulative database of alerts) with a Self-Serve version (that included a passive guide to evidence-based literature). Utilization of the service was the primary trial end-point. RESULTS: Mean logins to the library rose by 0.77 logins/month/user (95% CI 0.43, 1.11) in the Full-Serve group compared with the Self-Serve group. The proportion of Full-Serve participants who utilized the service during each month of the study period showed a sustained increase during the intervention period, with a relative increase of 57% (95% CI 12, 123) compared with the Self-Serve group. There were no differences in these proportions during the baseline period, and following the crossover of the Self-Serve group to Full-Serve, the Self-Serve group's usage became indistinguishable from that of the Full-Serve group (relative difference 4.4 (95% CI -23.7, 43.0). Also during the intervention and crossover periods, measures of self-reported usefulness did not show a difference between the 2 groups. CONCLUSION: A quality- and relevance-rated online literature service increased the utilization of evidence-based information from a digital library by practicing physicians.
OBJECTIVES: Systematic reviews of the literature are instrumental for bridging research to health care practice and are widely available through databases such as MEDLINE and EMBASE. Search strategies have been developed to aid users in MEDLINE, but no empirical work has been done for EMBASE. The objective of this study was to develop search strategies that optimize the retrieval of methodologically sound systematic reviews from EMBASE. STUDY DESIGN AND SETTING: An analytic survey was conducted, comparing hand searches of 55 journals with retrievals from EMBASE for 4,843 candidate search terms and 17,004 combinations. Candidate search strategies were run in EMBASE, the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated. RESULTS: Two hundred twenty (16.2%) of the 1,354 articles classified as a review met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 94.6% with specificity at 63.7%, whereas combinations of search terms to optimize specificity reached peak specificities of 99.3% with sensitivity at 61.4%. CONCLUSION: Empirically derived search strategies can achieve high sensitivity and specificity for retrieving methodologically sound systematic reviews from EMBASE.
BACKGROUND: Economic evaluations in the medical literature compare competing diagnosis or treatment methods for their use of resources and their expected outcomes. The best evidence currently available from research regarding both cost and economic comparisons will continue to expand as this type of information becomes more important in today's clinical practice. Researchers and clinicians need quick, reliable ways to access this information. A key source of this type of information is large bibliographic databases such as EMBASE. The objective of this study was to develop search strategies that optimize the retrieval of health costs and economics studies from EMBASE. METHODS: We conducted an analytic survey, comparing hand searches of journals with retrievals from EMBASE for candidate search terms and combinations. 6 research assistants read all issues of 55 journals indexed by EMBASE for the publishing year 2000. We rated all articles using purpose and quality indicators and categorized them into clinically relevant original studies, review articles, general papers, or case reports. The original and review articles were then categorized for purpose (i.e., cost and economics and other clinical topics) and depending on the purpose as 'pass' or 'fail' for methodologic rigor. Candidate search strategies were developed for economic and cost studies, then run in the 55 EMBASE journals, the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated. RESULTS: Combinations of search terms for detecting both cost and economic studies attained levels of 100% sensitivity with specificity levels of 92.9% and 92.3% respectively. When maximizing for both sensitivity and specificity, the combination of terms for detecting cost studies (sensitivity) increased 2.2% over the single term but at a slight decrease in specificity of 0.9%. The maximized combination of terms for economic studies saw no change in sensitivity from the single term and only a 0.1% increase in specificity. CONCLUSION: Selected terms have excellent performance in the retrieval of studies of health costs and economics from EMBASE.
CONTEXT: Most articles in clinical journals are not appropriate for direct application by individual clinicians. OBJECTIVE: To create a second order of clinical peer review for journal articles to determine which articles are most relevant for specific clinical disciplines. DESIGN AND SETTING: A 2-stage prospective observational study in which research staff reviewed all issues of over 110 (number has varied slightly as new journals were added or discarded from review but number has always been over 110) clinical journals and selected each article that met critical appraisal criteria from January 2003 through the present. Practicing physicians were recruited from around the world, excluding Northern Ontario, to the McMaster Online Rating of Evidence (MORE) system and registered as raters according to their clinical disciplines. An automated system assigned each qualifying article to raters for each pertinent clinical discipline, and recorded their online assessments of the articles on 7-point scales (highest score, 7) of relevance and newsworthiness (defined as useful new information for physicians). Rated articles fed an online alerting service, the McMaster Premium Literature Service (PLUS). Physicians from Northern Ontario were invited to register with PLUS and then receive e-mail alerts about articles according to MORE system peer ratings for their own discipline. Online access by PLUS users of PLUS alerts, raters' comments, article abstracts, and full-text journal articles was automatically recorded. MAIN OUTCOME MEASURES: Clinical rater recruitment and performance. Relevance and newsworthiness of journal articles to clinical practice in the discipline of the rating physician. RESULTS: Through October 2005, MORE had 2139 clinical raters, and PLUS had 5892 articles with 45 462 relevance ratings and 44 724 newsworthiness ratings collected since 2003. On average, clinicians rated systematic review articles higher for relevance to practice than articles with original evidence and lower for useful new information. Primary care physicians rated articles lower than did specialists (P<.05). Of the 98 physicians who registered for PLUS, 88 (90%) used it on 3136 occasions during an 18-month test period. CONCLUSIONS: This demonstration project shows the feasibility and use of a post-publication clinical peer review system that differentiates published journal articles according to the interests of a broad range of clinical disciplines.
OBJECTIVE: General practitioners, mental health practitioners, and researchers wishing to retrieve the best current research evidence in the content area of mental health may have a difficult time when searching large electronic databases such as MEDLINE. When MEDLINE is searched unaided, key articles are often missed while retrieving many articles that are irrelevant to the search. The objectives of this study were to develop optimal search strategies to detect articles with mental health content and to determine the effect of combining mental health content search strategies with methodologic search strategies calibrated to detect the best studies of treatment. METHOD: An analytic survey was conducted, comparing hand searches of 29 journals with retrievals from MEDLINE for 3,395 candidate search terms and 11,317 combinations. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated. RESULTS: 3,277 (26.8%) of the 12,233 articles classified in the 29 journals were considered to be of interest to the discipline area of mental health. Search term combinations reached peak sensitivities of 98.4% with specificity at 50.0%, whereas combinations of search terms to optimize specificity reached peak specificities of 97.1% with sensitivity at 51.7%. Combining content search strategies with methodologic search strategies for treatment led to improved precision: substantive decreases in the number of articles that needed to be sorted through in order to find target articles. CONCLUSION: Empirically derived search strategies can achieve high sensitivity and specificity for retrieving mental health content from MEDLINE. Combining content search strategies with methodologic search strategies led to more precise searches.
OBJECTIVE: A municipal water system became contaminated with Escherichia coli O157:H7 and Campylobacter spp. Beginning 2 years after an outbreak, all residents from the region were invited to participate in a cohort study assessing the risk of long-term sequelae. We aimed to develop a method to grade the accuracy and severity of self-reported acute symptoms. STUDY DESIGN AND SETTING: We corroborated participant survey responses with health records at the time of the outbreak. Of the 4,135 participants, 1,388 were asymptomatic during the outbreak, 1,752 had symptoms of acute self-limited gastroenteritis that could neither be confirmed nor refuted by prior health records, and 995 had symptoms that necessitated medical attention (and thus were confirmed by prior health records). RESULTS: The gradient related to the severity of acute symptoms. Compared to those with unconfirmed gastroenteritis, participants with confirmed gastroenteritis were more likely to describe fever, bloody diarrhea, and prolonged diarrhea (all P < .03). The gradient also correlated with long-term plausible outcomes, including chronic gastrointestinal symptoms, chronic symptoms of arthritis or depression, and the avoidance of municipal water ingestion after the outbreak (P for trend consistently < .03). Conversely, for the outcome of chronic tinnitus, an association was neither expected nor observed (P for trend = .26). CONCLUSION: We successfully characterized a gradient to be used in future primary analyses assessing the risk of long-term health sequelae after an outbreak.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
PURPOSE: To design optimal search strategies for locating sound therapy studies and review articles in CiNAHL in the year 2000. DESIGN AND METHODS: An analytic survey was conducted, comparing hand searches of 75 journals with retrievals from CINAHL for 5,020 candidate search terms and 17,900 combinations for therapy and 5,977 combinations for review articles. All articles were rated with purpose and quality indicators. Candidate search strategies were used in CINAHL, and the retrievals were compared with results of the hand searches. The proposed search strategies were treated as "diagnostic tests" for sound studies and the manual review of the literature was treated as the "gold standard." Operating characteristics of the search strategies were calculated. FINDINGS: Of the 1,383 articles about treatment, 506 (36.6%) met basic criteria for scientific merit and 127 (17.9%) of the 711 articles classified as a review met the criteria for systematic reviews. For locating sound treatment studies, a three-term strategy maximized sensitivity at 99.4% but with compromised specificity at 58.3%, and a two-term strategy maximized specificity at 98.5% but with compromised sensitivity at 52.0%. For detecting systematic reviews, a three-term strategy maximized sensitivity at 91.3% while keeping specificity high at 95.4%, and a single-term strategy maximized specificity at 99.6% but with compromised sensitivity at 42.5%. Three-term search strategies optimizing sensitivity and specificity achieved these values over 91% for detecting sound treatment studies and over 76% for detecting systematic reviews. CONCLUSIONS: Search strategies combining indexing terms and text words can achieve high sensitivity and specificity for retrieving sound treatment studies and review articles in CINAHL.
Explore the source record for details and available documents.
Researchers and practitioners have problems retrieving qualitative studies. Search strategies that can easily and effectively retrieve these studies from large databases such as PsycINFO are therefore important. To determine if search strategies can identify qualitative studies, 64 journals published in 2000 were hand searched using explicit methodological criteria to identify qualitative studies. The authors tested multiple search strategies using 4,985 potential search terms in PsycINFO (Ovid Technologies) and compared the results with the hand search data to calculate retrieval effectiveness. A total of 125 qualitative studies were identified. Single-term and multiple-term strategies had sensitivities (maximizing retrieval of qualitative studies) up to 94.4% and specificities (minimizing retrieval of nonqualitative studies and reports) up to 98.6% with ranges of precision and accuracy. Search strategies included terms that were variations of interview, qualitative, themes, and experience. Formal indexing terms performed poorly. Empirically derived search strategies combining textwords can effectively, but not perfectly, retrieve qualitative studies from PsycINFO.
Qualitative researchers address many issues relevant to patient health care. Their studies appear in an array of journals, making literature searching difficult. Large databases such as EMBASE provide a means of retrieving qualitative research, but these studies represent only a minuscule fraction of published articles, making electronic retrieval problematic. Little work has been done on developing search strategies for the detection of qualitative studies. The objective of this study was to develop optimal search strategies to retrieve qualitative studies in EMBASE for the 2000 publishing year. The authors conducted an analytic survey, comparing hand searches of journals with retrievals from EMBASE for candidate search terms and combinations. Search strategies reached peak sensitivities at 94.2% and peak specificities of 99.7%. Combining search terms to optimize the combination of sensitivity and specificity resulted in values over 89% for both. The authors identified search strategies with high performance for retrieving qualitative studies in EMBASE.
OBJECTIVE: The ability to accurately identify articles about therapy in large bibliographic databases such as EMBASE is important for researchers and clinicians. Our study aimed to develop optimal search strategies for detecting sound treatment studies in EMBASE in the year 2000. METHODS: Hand searches of journals were compared with retrievals from EMBASE for candidate search strategies. Six trained research assistants reviewed fifty-five journals indexed in EMBASE and rated articles using purpose and quality indicators. Candidate search strategies were developed for identifying treatment articles and then tested, and the retrievals were compared with the hand-search data. The operating characteristics of the strategies were calculated. RESULTS: Three thousand eight hundred fifty articles were original studies on treatment, of which 1,256 (32.6%) were methodologically sound. Combining search terms revealed a top performing strategy (random:.tw. OR clinical trial:.mp. OR exp health care quality) with sensitivity of 98.9% and specificity of 72.0%. Maximizing specificity, a top performing strategy (double-blind:.mp. OR placebo:.tw. OR blind: .tw.) achieved a value over 96.0%, but with compromised sensitivity at 51.7%. A 3-term strategy achieved the best optimization of sensitivity and specificity (random:.tw. OR placebo:.mp. OR double-blind:.tw.), with both these values over 92.0%. CONCLUSION: Search strategies can achieve high performance for retrieving sound treatment studies in EMBASE.