Comparison of top-performing search strategies for detecting clinically sound treatment studies and systematic reviews in MEDLINE and EMBASE.
Explore the source record for details and available documents.
Databases, Bibliographic↗
Biomedical subjects
R Brian Haynes
Publications and source records attributed to R Brian Haynes.
An overview of the design and methods for retrieving high-quality studies for clinical care.
BACKGROUND: With the information explosion, the retrieval of the best clinical evidence from large, general purpose, bibliographic databases such as MEDLINE can be difficult. Both researchers conducting systematic reviews and clinicians faced with a patient care question are confronted with the daunting task of searching for the best medical literature in electronic databases. Many have advocated the use of search filters or "hedges" to assist with the searching process. The purpose of this report is to describe the design and methods of a study that set out to develop optimal search strategies for retrieving sound clinical studies of health disorders in large electronics databases. OBJECTIVE: To describe the design and methods of a study that set out to develop optimal search strategies for retrieving sound clinical studies of health disorders in large electronic databases. DESIGN: An analytic survey comparing hand searches of 170 journals in the year 2000 with retrievals from MEDLINE, EMBASE, CINAHL, and PsycINFO for candidate search terms and combinations. The sensitivity, specificity, precision, and accuracy of unique search terms and combinations of search terms were calculated. CONCLUSION: A study design modeled after a diagnostic testing procedure with a gold standard (the hand search of the literature) and a test (the search terms) is an effective way of developing, testing, and validating search strategies for use in large electronic databases.
Databases, Bibliographic↗
Risk of hypertension and reduced kidney function after acute gastroenteritis from bacteria-contaminated drinking water.
BACKGROUND: The long-term health consequences of acute bacterial gastroenteritis remain uncertain. We studied the risk of hypertension and reduced kidney function after an outbreak of acute gastroenteritis due to contamination of a regional drinking water supply with Escherichia coli O157:H7 and Campylobacter species. METHODS: A total of 1958 adults with no known history of hypertension or kidney disease before the outbreak participated in a long-term follow-up study. Of the participants, 675 had been asymptomatic during the outbreak, 909 had had moderate symptoms of acute self-limited gastroenteritis, and 374 had had severe symptoms that necessitated medical attention. The outcomes of interest were a diagnosis of hypertension or the presence of reduced kidney function and albuminuria during the follow-up period. RESULTS: After a mean follow-up of 3.7 years after the outbreak, hypertension was diagnosed in 27.0% of participants who had been asymptomatic during the outbreak and in 32.3% and 35.9% of those who had had moderate and severe symptoms of acute gastroenteritis respectively (trend p = 0.009). Compared with the asymptomatic participants, those with moderate and severe symptoms of gastroenteritis had an adjusted relative risk of hypertension of 1.15 (95% confidence interval [CI] 0.97-1.35) and 1.28 (95% CI 1.04-1.56) respectively. A similar graded association was seen for reduced kidney function, defined as the presence of an estimated glomerular filtration rate below 60 mL/min per 1.73 m2 (trend p = 0.03). No association was observed between gastroenteritis and the subsequent risk of albuminuria. INTERPRETATION: Acute bacterial gastroenteritis necessitating medical attention was associated with an increased risk of hypertension and reduced kidney function 4 years after infection. Maintaining safe drinking water remains essential to human health, as transient bacterial contaminations may have implications well beyond a period of acute self-limited illness.
Acute Disease↗
Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey.
OBJECTIVE: To develop and test optimal Medline search strategies for retrieving sound clinical studies on prevention or treatment of health disorders. DESIGN: Analytical survey. DATA SOURCES: 161 clinical journals indexed in Medline for the year 2000. MAIN OUTCOME MEASURES: Sensitivity, specificity, precision, and accuracy of 4862 unique terms in 18 404 combinations. RESULTS: Only 1587 (24.2%) of 6568 articles on treatment met criteria for testing clinical interventions. Combinations of search terms reached peak sensitivities of 99.3% (95% confidence interval 98.7% to 99.8%) at a specificity of 70.4% (69.8% to 70.9%). Compared with best single terms, best multiple terms increased sensitivity for sound studies by 4.1% (absolute increase), but with substantial loss of specificity (absolute difference 23.7%) when sensitivity was maximised. When terms were combined to maximise specificity, 97.4% (97.3% to 97.6%) was achieved, about the same as that achieved by the best single term (97.6%, 97.4% to 97.7%). The strategies newly reported in this paper outperformed other validated search strategies except for two strategies that had slightly higher specificity (98.1% and 97.6% v 97.4%) but lower sensitivity (42.0% and 92.8% v 93.1%). CONCLUSION: New empirical search strategies have been validated to optimise retrieval from Medline of articles reporting high quality clinical studies on prevention or treatment of health disorders.
Information Storage and Retrieval↗
Optimal search strategies for identifying sound clinical prediction studies in EMBASE.
BACKGROUND: Clinical prediction guides assist clinicians by pointing to specific elements of the patient's clinical presentation that should be considered when forming a diagnosis, prognosis or judgment regarding treatment outcome. The numbers of validated clinical prediction guides are growing in the medical literature, but their retrieval from large biomedical databases remains problematic and this presents a barrier to their uptake in medical practice. We undertook the systematic development of search strategies ("hedges") for retrieval of empirically tested clinical prediction guides from EMBASE. METHODS: An analytic survey was conducted, testing the retrieval performance of search strategies run in EMBASE against the gold standard of hand searching, using a sample of all 27,769 articles identified in 55 journals for the 2000 publishing year. All articles were categorized as original studies, review articles, general papers, or case reports. The original and review articles were then tagged as 'pass' or 'fail' for methodologic rigor in the areas of clinical prediction guides and other clinical topics. Search terms that depicted clinical prediction guides were selected from a pool of index terms and text words gathered in house and through request to clinicians, librarians and professional searchers. A total of 36,232 search strategies composed of single and multiple term phrases were trialed for retrieval of clinical prediction studies. The sensitivity, specificity, precision, and accuracy of search strategies were calculated to identify which were the best. RESULTS: 163 clinical prediction studies were identified, of which 69 (42.3%) passed criteria for scientific merit. A 3-term strategy optimized sensitivity at 91.3% and specificity at 90.2%. Higher sensitivity (97.1%) was reached with a different 3-term strategy, but with a 16% drop in specificity. The best measure of specificity (98.8%) was found in a 2-term strategy, but with a considerable fall in sensitivity to 60.9%. All single term strategies performed less well than 2- and 3-term strategies. CONCLUSION: The retrieval of sound clinical prediction studies from EMBASE is supported by several search strategies.
Databases, Bibliographic↗
Optimal search strategies for detecting clinically sound prognostic studies in EMBASE: an analytic survey.
BACKGROUND: Clinical end users of EMBASE have a difficult time retrieving articles that are both scientifically sound and directly relevant to clinical practice. Search filters have been developed to assist end users in increasing the success of their searches. Many filters have been developed for the literature on therapy and reviews for use in MEDLINE, but little has been done for use in EMBASE with no filter development for studies of prognosis. The objective of this study was to determine how well various methodologic textwords, index terms, and their Boolean combinations retrieve methodologically sound literature on the prognosis of health disorders in EMBASE. METHODS: An analytic survey was conducted, comparing hand searches of 55 journals with retrievals from EMBASE for 4,843 candidate search terms and 8,919 combinations. All articles were rated using purpose and quality indicators, and clinically relevant prognostic articles were categorized as "pass" or "fail" according to explicit criteria for scientific merit. Candidate search strategies were run in EMBASE, the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated. RESULTS: Of the 1,064 articles about prognosis, 148 (13.9%) met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 98.7% with specificity at 50.6%. Compared with best single terms, best multiple terms increased sensitivity for sound studies by 12.2% (absolute increase), while decreasing specificity (absolute decrease 5.1%) when sensitivity was maximized. Combinations of search terms reached peak specificities of 93.4% with sensitivity at 50.7%. Compared with best single terms, best multiple terms increased specificity for sound studies by 7.1% (absolute increase), while decreasing sensitivity (absolute decrease 8.8%) when specificity was maximized. CONCLUSION: Empirically derived search strategies combining indexing terms and textwords can achieve high sensitivity or specificity for retrieving sound prognostic studies from EMBASE.
Abstracting and Indexing↗
EMBASE search strategies for identifying methodologically sound diagnostic studies for use by clinicians and researchers.
BACKGROUND: Accurate diagnosis by clinicians is the cornerstone of decision making for recommending clinical interventions. The current best evidence from research concerning diagnostic tests changes unpredictably as science advances. Both clinicians and researchers need dependable access to published evidence concerning diagnostic accuracy. Bibliographic databases such as EMBASE provide the most widely available entrée to this literature. The objective of this study was to develop search strategies that optimize the retrieval of methodologically sound diagnostic studies from EMBASE for use by clinicians. METHODS: An analytic survey was conducted, comparing hand searches of 55 journals with retrievals from EMBASE for 4,843 candidate search terms and 6,574 combinations. All articles were rated using purpose and quality indicators, and clinically relevant diagnostic accuracy articles were categorized as 'pass' or 'fail' according to explicit criteria for scientific merit. Candidate search strategies were run in EMBASE, the retrievals being compared with the hand search data. The proposed search strategies were treated as "diagnostic tests" for sound studies and the manual review of the literature was treated as the "gold standard." The sensitivity, specificity, precision and accuracy of the search strategies were calculated. RESULTS: Of the 433 articles about diagnostic tests, 97 (22.4%) met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 100% with specificity at 70.4%. Compared with best single terms, best multiple terms increased sensitivity for sound studies by 8.2% (absolute increase), but decreased specificity (absolute decrease 6%) when sensitivity was maximized. When terms were combined to maximize specificity, the single term "specificity.tw." (specificity of 98.2%) outperformed combinations of terms. CONCLUSION: Empirically derived search strategies combining indexing terms and textwords can achieve high sensitivity and specificity for retrieving sound diagnostic studies from EMBASE. These search filters will enhance the searching efforts of clinicians.
Abstracting and Indexing↗
Developing optimal search strategies for detecting clinically sound and relevant causation studies in EMBASE.
BACKGROUND: Evaluating the existence and strength of an association between a putative cause and adverse clinical outcome is complex and best done by assessing all available evidence. With the increasing burden of chronic disease, greater time demands on health professionals, and the explosion of information, effective retrieval of best evidence has become both more important and more difficult. Optimal search retrieval can be hampered by a number of obstacles, especially poor search strategies, but using empirically tested methodological search filters can enhance the accuracy of searches for sound evidence concerning etiology. Although such filters have previously been developed for studies of relevance to causation in MEDLINE, no empirically tested search strategy exists for EMBASE. METHODS: An analytic survey was conducted, comparing hand searches of journals with retrievals from EMBASE for candidate search terms and combinations. 6 research assistants read all issues of 55 journals indexed in EMBASE. All articles were rated using purpose and quality indicators and categorized into clinically relevant original studies, review articles, general papers, or case reports. The original and review articles were then categorized as 'pass' or 'fail' for scientific merit according to explicit criteria in the areas of causation (etiology) and other clinical topics. Candidate search strategies were developed for causation, then run in a subset of 55 EMBASE journals, the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated. RESULTS: Of the 1489 studies classified as causation, 14% were methodologically sound. When search terms were combined, sensitivity reached 92%. Compared with the best single-term strategy, the best combination of terms resulted in an absolute increase in sensitivity (19%) and specificity (5.2%). Maximizing specificity for combined terms resulted in an increase of 7.1% compared with the single term but this came at an expense of sensitivity (39% absolute decrease). A search strategy that optimized the trade-off between sensitivity and specificity achieved 81.9% for sensitivity and 81.4% for specificity. CONCLUSION: We have discovered search strategies that retrieve high quality studies of causation from EMBASE with high sensitivity, high specificity, or an optimal balance of each.
Causality↗
Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review.
CONTEXT: Developers of health care software have attributed improvements in patient care to these applications. As with any health care intervention, such claims require confirmation in clinical trials. OBJECTIVES: To review controlled trials assessing the effects of computerized clinical decision support systems (CDSSs) and to identify study characteristics predicting benefit. DATA SOURCES: We updated our earlier reviews by searching the MEDLINE, EMBASE, Cochrane Library, Inspec, and ISI databases and consulting reference lists through September 2004. Authors of 64 primary studies confirmed data or provided additional information. STUDY SELECTION: We included randomized and nonrandomized controlled trials that evaluated the effect of a CDSS compared with care provided without a CDSS on practitioner performance or patient outcomes. DATA EXTRACTION: Teams of 2 reviewers independently abstracted data on methods, setting, CDSS and patient characteristics, and outcomes. DATA SYNTHESIS: One hundred studies met our inclusion criteria. The number and methodologic quality of studies improved over time. The CDSS improved practitioner performance in 62 (64%) of the 97 studies assessing this outcome, including 4 (40%) of 10 diagnostic systems, 16 (76%) of 21 reminder systems, 23 (62%) of 37 disease management systems, and 19 (66%) of 29 drug-dosing or prescribing systems. Fifty-two trials assessed 1 or more patient outcomes, of which 7 trials (13%) reported improvements. Improved practitioner performance was associated with CDSSs that automatically prompted users compared with requiring users to activate the system (success in 73% of trials vs 47%; P = .02) and studies in which the authors also developed the CDSS software compared with studies in which the authors were not the developers (74% success vs 28%; respectively, P = .001). CONCLUSIONS: Many CDSSs improve practitioner performance. To date, the effects on patient outcomes remain understudied and, when studied, inconsistent.
Clinical Pharmacy Information Systems↗
How to appraise the effectiveness of treatment.
Before implementing a new therapy, we should ascertain the benefits and risks of the therapy and assure ourselves that the resources consumed during the intervention will not be exorbitant. In the hierarchy of research designs, the results of randomized controlled trials, especially if systematically reviewed, are considered the highest level of evidence. We suggest a three-step approach to using an article from the medical literature to guide your patient care. We recommend that readers ask whether the study can provide valid results, review the results, and consider how the results can be applied to patient care. Given the time constraints of busy surgical practices and surgical training programs, applying this analysis to every relevant article will be challenging. However, the basis of this process is essentially what we all do many times each week when making decisions about whether and how to treat patients. Making this process explicit with guidelines to assess the strength of the available evidence can serve to improve patient care. It also allow us to defend therapeutic interventions based on available evidence and not anecdote.
Evaluation Studies as Topic↗
Microalbuminuria three years after recovery from Escherichia coli O157 hemolytic uremic syndrome due to municipal water contamination.
BACKGROUND: Knowledge of the long-term renal prognosis of diarrhea associated hemolytic uremic syndrome (HUS) is important for patient counseling and follow-up. However, estimates of long-term risk are highly variable, with previous studies not using a healthy control group. METHODS: A municipal water system in the small rural town of Walkerton, Ontario, became contaminated with Escherichia coli O157:H7 in 2000. A cohort of 19 children who recovered from HUS was randomly age- and sex-matched to 38 children with no symptoms at the time of the outbreak. Both groups had detailed renal function testing 3 years after the outbreak, including a random urine albumin to creatinine, glomerular filtration rate estimated by Schwartz formula, and automated and manual blood pressure measurements. RESULTS: There were no baseline differences between the groups with respect to age (mean 4.8 years, range 1 to 15), sex, or birth weight (mean 3.4 kg). In follow-up there were no differences between the groups in body surface area (mean 1.0 m(2)), or in the methods by which renal function was assessed. Compared to the group with no symptoms, patients with HUS demonstrated more microalbuminuria [32% vs. 5%, relative risk 4.8 (95% CI 1.1 to 22.0)], a nonsignificant trend toward lower GFR (124 vs. 134 mL/min per 1.73 m(2)), and no difference in blood pressure. CONCLUSION: Children may demonstrate microalbuminuria 3 years after recovering from HUS. Longer follow-up is needed to determine if this finding has clinical relevance and utility.
Albuminuria↗
Of studies, summaries, synopses, and systems: the "4S" evolution of services for finding current best evidence.
Explore the source record for details and available documents.
Evidence-Based Medicine↗
bmjupdates+, a new free service for evidence-based clinical practice.
Explore the source record for details and available documents.
Databases, Bibliographic↗
McMaster Premium Literature Service (PLUS): an evidence-based medicine information service delivered on the Web.
BACKGROUND: Evidence-based medicine (EBM) attempts to narrow the gap between knowledge and practice, but ready access to evidence-based resources remains a challenge to practicing physicians. OBJECTIVE: To describe a new, EBM information delivery service, its trial design, and baseline data. METHODS: McMaster PLUS (Premium Literature Service), composed of a continuously updated database and web-based interface, delivers scientifically rigorous and clinically relevant research literature matched to individual physicians' clinical interests. A cluster randomized controlled trial is currently underway, comparing 2 versions of the PLUS system. RESULTS: As of Feb 2005, the PLUS database contained over 5700 scientifically sound, clinically relevant articles, published from 2001 to present. 68% of articles have full text links. Over 200 physicians in Northern Ontario have been randomized to one of 2 PLUS interfaces. CONCLUSION: McMaster PLUS has been designed to aid physicians to home in on high quality research that is highly relevant and important to their own clinical practice.
Databases as Topic↗
Optimal search strategies for retrieving systematic reviews from Medline: analytical survey.
OBJECTIVE: To develop optimal search strategies in Medline for retrieving systematic reviews. DESIGN: Analytical survey. DATA SOURCES: 161 journals published in 2000 indexed in Medline. MAIN OUTCOME MEASURES: The sensitivity, specificity, and precision of retrieval of systematic reviews of 4862 unique terms in 782,485 combinations of one to five terms were determined by comparison with a hand search of all articles (the criterion standard) in 161 journals published during 2000 (49,028 articles). RESULTS: Only 753 (1.5%) of the 49,028 articles were systematic reviews. The most sensitive strategy included five terms and had a sensitivity of 99.9% (95% confidence interval 99.6% to 100%) and a specificity of 52% (51.6% to 52.5%). The strategy that best minimised the difference between sensitivity and specificity had a sensitivity of 98% (97% to 99%) and specificity of 90.8% (90.5% to 91.1%). Highest precision for multiterm strategies, 57% (54% to 60%), was achieved at a sensitivity of 71% (68% to 74%). The term "cochrane database of systematic reviews.jn." was the most precise single term search strategy (sensitivity of 56% (52% to 60%) and precision of 96% (94% to 98%)). These strategies are available through the "limit" screen of Ovid's search interface for Medline. CONCLUSIONS: Systematic reviews can be retrieved from Medline with close to perfect sensitivity or specificity, or with high precision, by using empirical search strategies.
Information Storage and Retrieval↗
Optimal search strategies for detecting health services research studies in MEDLINE.
BACKGROUND: Evidence from health services research (HSR) is currently thinly spread through many journals, making it difficult for health services researchers, managers and policy-makers to find research on clinical practice guidelines and the appropriateness, process, outcomes, cost and economics of health care services. We undertook to develop and test search terms to retrieve from the MEDLINE database HSR articles meeting minimum quality standards. METHODS: The retrieval performance of 7445 methodologic search terms and phrases in MEDLINE (the test) were compared with a hand search of the literature (the gold standard) for each issue of 68 journal titles for the year 2000 (a total of 25,936 articles). We determined sensitivity, specificity and precision (the positive predictive value) of the MEDLINE search strategies. RESULTS: A majority of the articles that were classified as outcome assessment, but fewer than half of those in the other categories, were considered methodologically acceptable (no methodologic criteria were applied for cost studies). Combining individual search terms to maximize sensitivity, while keeping specificity at 50% or more, led to sensitivities in the range of 88.1% to 100% for several categories (specificities ranged from 52.9% to 97.4%). When terms were combined to maximize specificity while keeping sensitivity at 50% or more, specificities of 88.8% to 99.8% were achieved. When terms were combined to maximize sensitivity and specificity while minimizing the differences between the 2 measurements, most strategies for HSR categories achieved sensitivity and specificity of at least 80%. INTERPRETATION: Sensitive and specific search strategies were validated for retrieval of HSR literature from MEDLINE. These strategies have been made available for public use by the US National Library of Medicine at www.nlm.nih.gov/nichsr/hedges/search.html.
Costs and Cost Analysis↗
Author self-citation in the diabetes literature.
BACKGROUND: Author self-citation is the practice of citing one's previous publications in a new publication. Its extent is unknown. We studied author self-citation, choosing the major clinical field of diabetes mellitus to represent the general medical literature. METHODS: We identified every article about diabetes mellitus in 170 hand-searched clinical journals published in 2000. For every article, we recorded the bibliographic citation and publication type (original or review article) and assessed the methodologic rigour. Citation information was obtained from the ISI Web of Knowledge in April 2003. RESULTS: Of 49,028 articles, 289 were about diabetes mellitus and had citation information. Citation counts ranged from 0 to 347 (median 6, interquartile range [IQR] 2-12). Author self-citation counts ranged from 0 to 16 (median 1, IQR 0-2). Author self-citations accounted for an average of 18% (95% confidence interval [CI] 15%-21%) and a median of 7% (95% CI 5%- 11%) of all citations of each publication that was cited at least once (n = 266). Original articles had double the mean proportion of author self-citations compared with review articles (19% v. 9%; median 7% v. 0%, difference 7%, 95% CI 0- 10%). Methodologic rigour and review type were not significantly associated with subsequent author self-citation. INTERPRETATION: Nearly one-fifth of all citations to articles about diabetes mellitus in clinical journals in the year 2000 were author self-citations. The frequency of self-citation was not associated with the quality of publications. These findings are likely applicable to the general clinical medicine literature and may have important implications for the assessment of journal or publication importance and the process of scientific discovery.
Authorship↗
Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE: an analytic survey.
BACKGROUND: Clinical end users of MEDLINE have a difficult time retrieving articles that are both scientifically sound and directly relevant to clinical practice. Search filters have been developed to assist end users in increasing the success of their searches. Many filters have been developed for the literature on therapy and reviews but little has been done in the area of prognosis. The objective of this study is to determine how well various methodologic textwords, Medical Subject Headings, and their Boolean combinations retrieve methodologically sound literature on the prognosis of health disorders in MEDLINE. METHODS: An analytic survey was conducted, comparing hand searches of journals with retrievals from MEDLINE for candidate search terms and combinations. Six research assistants read all issues of 161 journals for the publishing year 2000. All articles were rated using purpose and quality indicators and categorized into clinically relevant original studies, review articles, general papers, or case reports. The original and review articles were then categorized as 'pass' or 'fail' for methodologic rigor in the areas of prognosis and other clinical topics. Candidate search strategies were developed for prognosis and run in MEDLINE - the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated. RESULTS: 12% of studies classified as prognosis met basic criteria for scientific merit for testing clinical applications. Combinations of terms reached peak sensitivities of 90%. Compared with the best single term, multiple terms increased sensitivity for sound studies by 25.2% (absolute increase), and increased specificity, but by a much smaller amount (1.1%) when sensitivity was maximized. Combining terms to optimize both sensitivity and specificity achieved sensitivities and specificities of approximately 83% for each. CONCLUSION: Empirically derived search strategies combining indexing terms and textwords can achieve high sensitivity and specificity for retrieving sound prognostic studies from MEDLINE.
Information Storage and Retrieval↗