Biomedical subjects

Elmer V Bernstam

Publications and source records attributed to Elmer V Bernstam.

12 recordsLinked to original sources

PubMed2007

A day in the life of PubMed: analysis of a typical day's query log.

OBJECTIVE: To characterize PubMed usage over a typical day and compare it to previous studies of user behavior on Web search engines. DESIGN: We performed a lexical and semantic analysis of 2,689,166 queries issued on PubMed over 24 consecutive hours on a typical day. MEASUREMENTS: We measured the number of queries, number of distinct users, queries per user, terms per query, common terms, Boolean operator use, common phrases, result set size, MeSH categories, used semantic measurements to group queries into sessions, and studied the addition and removal of terms from consecutive queries to gauge search strategies. RESULTS: The size of the result sets from a sample of queries showed a bimodal distribution, with peaks at approximately 3 and 100 results, suggesting that a large group of queries was tightly focused and another was broad. Like Web search engine sessions, most PubMed sessions consisted of a single query. However, PubMed queries contained more terms. CONCLUSION: PubMed's usage profile should be considered when educating users, building user interfaces, and developing future biomedical information retrieval systems.

Algorithms↗

PubMed2006

Toward a veterinary informatics research agenda: an analysis of the PubMed-indexed literature.

PURPOSE: Veterinary medicine and human health are inextricably intertwined. Effective tracking of veterinary information - veterinary informatics - impacts not only veterinary medicine, but also public health, informatics research, and clinical care. However, veterinary informatics has received little attention from the general biomedical informatics community. METHODS: To identify both active and under-researched areas in veterinary informatics, we retrieved Medical Subject Heading (MeSH) descriptors for veterinary informatics-related citations and analyzed them by topic category, animal type, and journal. RESULTS: We found that the categories of veterinary informatics with the most growth were information/bibliographical retrieval, hardware/programming, and radiology/imaging. Less than two articles per year were published in the areas of computerized veterinary medical records, clinical decision support, standards, and controlled vocabularies. Veterinary informatics articles primarily address production animals such as cattle and sheep, and companion animals such as cats and dogs. Six journals account for 31% of the veterinary informatics literature, 35 journals account for 66%. CONCLUSIONS: Veterinary informatics remains an embryonic field with relatively few publications. With the exception of radiology/imaging, published articles are primarily focused on non-clinical areas such as hardware/programming and information retrieval. There are very few publications on controlled vocabularies, standards, methodologies for integrating disparate systems, computerized medical records, clinical decision support systems, and system usability. The lack of publications in these areas may hamper efforts to collect and track animal health data at a time when such data are potentially critical to human health.

Medical Informatics↗

PubMed2006

Context, automated decision support, and clinical practice guidelines: does the literature apply to the United States practice environment?

BACKGROUND: Context - the combined effect of factors such as physician type, clinical setting, and guideline characteristics - influences the ability of automated decision support (ADS) to improve physician compliance with clinical practice guidelines (CPGs). Our goal was to determine whether research about the utility of ADS for promoting CPG compliance is contextually applicable to United States physicians. METHODS: We extracted information about physicians, settings, and guidelines from all articles published in the last 10 years that describe original research about the use of ADS to promote CPG compliance. The extracted information was compared to the range of practice contexts seen in the United States. RESULTS: Nearly two-thirds (65.3%) of papers described studies conducted in an academic setting, but only 11% of physicians report academic affiliations (p<0.0001). Physician reimbursement structure is often not reported. Salaried physicians were explicitly included as subjects in 14% of articles, but make up 45% of US physicians (p<0.0001). There are little data about the generalizability of ADS research to emergency care settings (6% of articles), and nursing home or skilled nursing facilities (10% of articles). Finally, ADS has not been studied at all in several epidemiologically important disease categories. CONCLUSION: The literature does not adequately address some physician, setting, and guideline contexts. Before making policy or spending decisions based on the effectiveness of ADS, additional research is needed to determine whether ADS research can be generalized to under-represented contexts.

Decision Support Systems, Clinical↗

PubMed2006

Accuracy and self correction of information received from an internet breast cancer list: content analysis.

OBJECTIVES: To determine the prevalence of false or misleading statements in messages posted by internet cancer support groups and whether these statements were identified as false or misleading and corrected by other participants in subsequent postings. DESIGN: Analysis of content of postings. SETTING: Internet cancer support group Breast Cancer Mailing List. MAIN OUTCOME MEASURES: Number of false or misleading statements posted from 1 January to 23 April 2005 and whether these were identified and corrected by participants in subsequent postings. RESULTS: 10 of 4600 postings (0.22%) were found to be false or misleading. Of these, seven were identified as false or misleading by other participants and corrected within an average of four hours and 33 minutes (maximum, nine hours and nine minutes). CONCLUSIONS: Most posted information on breast cancer was accurate. Most false or misleading statements were rapidly corrected by participants in subsequent postings.

Breast Neoplasms↗

PubMed2006

Using hit curves to compare search algorithm performance.

Databases continue to grow but the metrics available to evaluate information retrieval systems have not changed. Large collections such as MEDLINE and the World Wide Web contain many relevant documents for common queries. Ranking is therefore increasingly important and successful information retrieval systems, such as Google, have emphasized ranking. However, existing evaluation metrics such as precision and recall, do not directly account for ranking. This paper describes a novel way of measuring information retrieval performance using weighted hit curves adapted from the field of statistical detection to reflect multiple desirable characteristics such as relevance, importance, and methodologic quality. In statistical detection, hit curves have been proposed to represent occurrence of interesting events during a detection process. Similarly, hit curves can be used to study the position of relevant documents within large result sets. We describe hit curves in light of a formal model of information retrieval, show how hit curves represent system performance including ranking, and define ways to statistically compare performance of multiple systems using hit curves. We provide example scenarios where traditional measures are less suitable than hit curves and conclude that hit curves may be useful for evaluating retrieval from large collections where ranking performance is crucial.

Algorithms↗

PubMed2005

Using citation data to improve retrieval from MEDLINE.

OBJECTIVE: To determine whether algorithms developed for the World Wide Web can be applied to the biomedical literature in order to identify articles that are important as well as relevant. DESIGN AND MEASUREMENTS A direct comparison of eight algorithms: simple PubMed queries, clinical queries (sensitive and specific versions), vector cosine comparison, citation count, journal impact factor, PageRank, and machine learning based on polynomial support vector machines. The objective was to prioritize important articles, defined as being included in a pre-existing bibliography of important literature in surgical oncology. RESULTS Citation-based algorithms were more effective than noncitation-based algorithms at identifying important articles. The most effective strategies were simple citation count and PageRank, which on average identified over six important articles in the first 100 results compared to 0.85 for the best noncitation-based algorithm (p < 0.001). The authors saw similar differences between citation-based and noncitation-based algorithms at 10, 20, 50, 200, 500, and 1,000 results (p < 0.001). Citation lag affects performance of PageRank more than simple citation count. However, in spite of citation lag, citation-based algorithms remain more effective than noncitation-based algorithms. CONCLUSION Algorithms that have proved successful on the World Wide Web can be applied to biomedical information retrieval. Citation-based algorithms can help identify important articles within large sets of relevant results. Further studies are needed to determine whether citation-based algorithms can effectively meet actual user information needs.

Algorithms↗

PubMed2005

Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed?

PURPOSE: Many criteria have been developed to rate the quality of online health information. To effectively evaluate quality, consumers must use quality criteria that can be reliably assessed. However, few instruments have been validated for inter-rater agreement. Therefore, we assessed the degree to which two raters could reliably assess 22 popularly cited quality criteria on a sample of 42 complementary and alternative medicine Web sites. METHODS: We determined the degree of inter-rater agreement by calculating the percentage agreement, Cohen's kappa, and prevalence- and bias-adjusted kappa (PABAK). RESULTS: Our un-calibrated analysis showed poor inter-rater agreement on eight of the 22 quality criteria. Therefore, we created operational definitions for each of the criteria, decreased the number of assessment choices and defined where to look for the information. As a result 18 of the 22 quality criteria were reliably assessed (inter-rater agreement > or = 0.6). CONCLUSIONS: We conclude that even with precise definitions, some commonly used quality criteria cannot be reliably assessed. However, inter-rater agreement can be improved with precise operational definitions.

Complementary Therapies↗

PubMed2005

Instruments to assess the quality of health information on the World Wide Web: what can our patients actually use?

OBJECTIVE: To find and assess quality-rating instruments that can be used by health care consumers to assess websites displaying health information. DATA SOURCES: Searches of PubMed, the World Wide Web (using five different search engines), reference tracing from identified articles, and a review of the of the American Medical Informatics Association's annual symposium proceedings. REVIEW METHODS: Sources were examined for availability, number of elements, objectivity, and readability. RESULTS: A total of 273 distinct instruments were found and analyzed. Of these, 80 (29%) made evaluation criteria publicly available and 24 (8.7%) had 10 or fewer elements (items that a user has to assess to evaluate a website). Seven instruments consisted of elements that could all be evaluated objectively. Of these seven, one instrument consisted entirely of criteria with acceptable interobserver reliability (kappa> or =0.6); another instrument met readability standards. CONCLUSIONS: There are many quality-rating instruments, but few are likely to be practically usable by the intended audience.

Humans↗

PubMed2005

Searching for cancer-related information online: unintended retrieval of complementary and alternative medicine information.

PURPOSE: The Web is an important source of health information for consumers. Use of complementary and alternative medicine (CAM) is also increasing. Therefore, we studied the likelihood that consumers will incidentally encounter CAM information while searching the Web and the factors that influence retrieval of CAM information. METHODS: We evaluated results retrieved by 10 cancer-related searches on six common search engines. RESULTS: Of 1121 search results, 16.2% displayed CAM information. Sponsored (i.e., paid) results were more likely to display CAM information than non-sponsored results (38% versus 7.5%, p < 0.001). In Overture and Google, sponsored results accounted for 51% and 39% of results on the first page. These search engines also retrieved more CAM web pages. Search engines distinguished sponsored and non-sponsored results, but disclosure statements describing the differences were confusing. Cancer type used as the search keyword did not influence the number of CAM web pages retrieved. However, synonyms of cancer differed in their retrieval of CAM web pages (p < 0.001). Consistent with prior studies of Web search engine overlap, we found that 28% of CAM results were retrieved by two or more search engines. CONCLUSIONS: Clinicians should help consumers recognize sponsored results and encourage search engines to clearly explain sponsored results.

Complementary Therapies↗

PubMed2005

Using incomplete citation data for MEDLINE results ranking.

Information overload is a significant problem for modern medicine. Searching MEDLINE for common topics often retrieves more relevant documents than users can review. Therefore, we must identify documents that are not only relevant, but also important. Our system ranks articles using citation counts and the PageRank algorithm, incorporating data from the Science Citation Index. However, citation data is usually incomplete. Therefore, we explore the relationship between the quantity of citation information available to the system and the quality of the result ranking. Specifically, we test the ability of citation count and PageRank to identify "important articles" as defined by experts from large result sets with decreasing citation information. We found that PageRank performs better than simple citation counts, but both algorithms are surprisingly robust to information loss. We conclude that even an incomplete citation database is likely to be effective for importance ranking.

Algorithms↗

PubMed2004

Efficacy of quality criteria to identify potentially harmful information: a cross-sectional survey of complementary and alternative medicine web sites.

BACKGROUND: Many users search the Internet for answers to health questions. Complementary and alternative medicine (CAM) is a particularly common search topic. Because many CAM therapies do not require a clinician's prescription, false or misleading CAM information may be more dangerous than information about traditional therapies. Many quality criteria have been suggested to filter out potentially harmful online health information. However, assessing the accuracy of CAM information is uniquely challenging since CAM is generally not supported by conventional literature. OBJECTIVE: The purpose of this study is to determine whether domain-independent technical quality criteria can identify potentially harmful online CAM content. METHODS: We analyzed 150 Web sites retrieved from a search for the three most popular herbs: ginseng, ginkgo and St. John's wort and their purported uses on the ten most commonly used search engines. The presence of technical quality criteria as well as potentially harmful statements (commissions) and vital information that should have been mentioned (omissions) was recorded. RESULTS: Thirty-eight sites (25%) contained statements that could lead to direct physical harm if acted upon. One hundred forty five sites (97%) had omitted information. We found no relationship between technical quality criteria and potentially harmful information. CONCLUSIONS: Current technical quality criteria do not identify potentially harmful CAM information online. Consumers should be warned to use other means of validation or to trust only known sites. Quality criteria that consider the uniqueness of CAM must be developed and validated.

Complementary Therapies↗

PubMed2002

Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites.

OBJECTIVES: To determine the characteristics of popular breast cancer related websites and whether more popular sites are of higher quality. DESIGN: The search engine Google was used to generate a list of websites about breast cancer. Google ranks search results by measures of link popularity---the number of links to a site from other sites. The top 200 sites returned in response to the query "breast cancer" were divided into "more popular" and "less popular" subgroups by three different measures of link popularity: Google rank and number of links reported independently by Google and by AltaVista (another search engine). MAIN OUTCOME MEASURES: Type and quality of content. RESULTS: More popular sites according to Google rank were more likely than less popular ones to contain information on ongoing clinical trials (27% v 12%, P=0.01 ), results of trials (12% v 3%, P=0.02), and opportunities for psychosocial adjustment (48% v 23%, P<0.01). These characteristics were also associated with higher number of links as reported by Google and AltaVista. More popular sites by number of linking sites were also more likely to provide updates on other breast cancer research, information on legislation and advocacy, and a message board service. Measures of quality such as display of authorship, attribution or references, currency of information, and disclosure did not differ between groups. CONCLUSIONS: Popularity of websites is associated with type rather than quality of content. Sites that include content correlated with popularity may best meet the public's desire for information about breast cancer.

Breast Neoplasms↗