Data mining in pharmacovigilance: lessons from phantom ships.
Explore the source record for details and available documents.
Biomedical subjects
Publications and source records attributed to Lester Reich.
Explore the source record for details and available documents.
PURPOSE: A population-based study and anecdotal reports have indicated that the publication of the Randomized Aldactone Evaluation Study (RALES) was associated with not merely a broader use of spironolactone in the treatment of heart failure, but also with a coinciding sharp increase in hyperkalemia-associated morbidity/mortality in patients also being treated with ACE-inhibitors. Data mining algorithms (DMAs) are being applied to spontaneous reporting system (SRS) databases in hopes of obtaining early warnings/additional insights into post-licensure safety data. We applied two DMAs (i.e. multi-item gamma Poisson shrinker [MGPS] and proportional reporting ratios [PRRs]) to spontaneous reporting system (SRS) data to determine if these DMAs could have provided an earlier indication of a possible hyperkalemia safety issue. METHODS: MGPS and PRRs were retrospectively applied to US FDA-AERS, an SRS database. Year-by-year analysis and analysis of increasing cumulative time intervals were performed on cases in which both spironolactone and hyperkalemia and possibly related cardiac events had been reported. RESULTS: Neither of the DMAs initially provided a compelling signal of disproportionate reporting (SDR) for hyperkalemia after publication of RALES. However, using events consistent with clinical sequelae of hyperkalemia (e.g,. sudden death), SDRs were identified with PRRs. CONCLUSIONS: The quality and usefulness of data mining analysis is highly situation dependent and may vary with the knowledge and experience of the drug safety reviewer. Our analysis suggests that contemporary DMAs may have significant limitations in detecting increased frequency of labeled events in real-life prospective pharmacovigilance. There is a paucity of research in this area and we recommend further research for new approaches to detecting increased frequency of labeled events.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
OBJECTIVE: To apply two data mining algorithms (DMAs) to Food and Drug Administration (FDA) Adverse Event Reporting System (AERS) reports that involved endotoxin-like reactions with intravenous gentamicin to determine whether a signal of disproportionate reporting of these events would have been generated concurrently with surveillance based on clinical observation. DESIGN: Multi-item gamma-Poisson shrinker (MGPS) and proportional reporting ratios (PRRs) were used. Data used for data mining consisted of an extract of the FDA AERS database. Previously published details of clusters of endotoxin-like reactions to intravenous gentamicin were used to select adverse events for data mining. RESULTS: The first signal of disproportionate reporting with any relevant event occurred in 1998, the year in which the outbreak was identified and evaluated by the Centers for Disease Control and Prevention and the FDA. In 1997, there were only 6 reports of rigors in the AERS; this jumped to 68 in 1998. In 1998, a signal was generated for endotoxic shock with PRRs but not with MGPS, based on one case. CONCLUSIONS: The two DMAs generated signals concurrently with the influx of reports. It would have been difficult for safety reviewers to ignore an increase in rigors by traditional methods of safety surveillance; therefore, DMAs might not have had a great deal to offer in this instance. If data mining were considered as a second-line defense to diligent clinical observations under similar circumstances, simple disproportionality methods such as PRRs might be more useful than DMAs such as MGPS when commonly cited thresholds are used.
Explore the source record for details and available documents.
The objective of this study was to apply 2 data-mining algorithms to a drug safety database to determine if these methods would have flagged potentially fatal/disabling adverse drug reactions that triggered black box warnings/drug withdrawals in advance of initial identification via "traditional" methods. Relevant drug-event combinations were identified from a journal publication. Data-mining algorithms using commonly cited disproportionality thresholds were then applied to the US Food and Drug Administration database. Seventy drug-event combinations were considered sufficiently specific for retrospective data mining. In a minority of instances, potential signals of disproportionate reporting were provided clearly in advance of initial identification via traditional pharmacovigilance methods. Data-mining algorithms have the potential to improve pharmacovigilance screening; however, for the majority of drug-event combinations, there was no substantial benefit of either over traditional methods. They should be considered as potential supplements to, and not substitutes for, traditional pharmacovigilance strategies. More research and experience will be needed to optimize deployment of data-mining algorithms in pharmacovigilance.
Data mining is receiving considerable attention as a tool for pharmacovigilance and is generating many perspectives on its uses. This paper presents four concepts that have appeared in various professional venues and represent potential sources of misunderstanding and/or entail extended discussions: (i) data mining algorithms are unvalidated; (ii) data mining algorithms allow data miners to objectively screen spontaneous report data; (iii) mathematically more complex Bayesian algorithms are superior to frequentist algorithms; and (iv) data mining algorithms are not just for hypothesis generation. Key points for a balanced perspective are that: (i) validation exercises have been done but lack a gold standard for comparison and are complicated by numerous nuances and pitfalls in the deployment of data mining algorithms. Their performance is likely to be highly situation dependent; (ii) the subjective nature of data mining is often underappreciated; (iii) simpler data mining models can be supplemented with 'clinical shrinkage', preserving sensitivity; and (iv) applications of data mining beyond hypothesis generation are risky, given the limitations of the data. These extended applications tend to 'creep', not pounce, into the public domain, leading to potential overconfidence in their results. Most importantly, in the enthusiasm generated by the promise of data mining tools, users must keep in mind the limitations of the data and the importance of clinical judgment and context, regardless of statistical arithmetic. In conclusion, we agree that contemporary data mining algorithms are promising additions to the pharmacovigilance toolkit, but the level of verification required should be commensurate with the nature and extent of the claimed applications.
PURPOSE: Several data mining algorithms (DMAs) are being studied in hopes of enhancing screening of large post-marketing safety databases for signals of novel adverse events (AEs). The objective of this study was to apply two DMAs to the United States FDA Adverse Event Reporting System (AERS) database to see whether signals of potentially fatal AEs with cancer drugs might have been identified earlier than with traditional methods. METHODS: Screening algorithms used for analysis were the multi-item gamma Poisson shrinker (MGPS) and proportional reporting ratios (PRRs). Data mining was performed on data from the FDA AERS database. When a signal was identified, it was compared with that in the year in which the event was added to package insert and/or the year a "case series" was published. A recent publication summarizing the time of dissemination of information on potentially fatal AEs to cancer drugs provided the data set for analysis. RESULTS: The peer-reviewed published analysis contained 21 drugs and 26 drug-event combinations (DECs) that were considered sufficiently specific for data mining. Twenty-four of the DECs generated a signal of disproportionate reporting with PRRs (6 at 1 year and 16 from 2 years to 18 years prior to either a published "case series" or a package insert change) and 20 with MGPS (3 at 1 year and 11 from 2 years to 16 years prior to either a published "case series" or a package insert change). Two DECs did not signal with either DMA. CONCLUSION: At least one commonly cited DMA generated a signal of disproportionate reporting for 24 of 26 DECs for selected cancer drugs. For 16 DECs, one could conclude that a signal was generated well in advance (> or =2 years) of standard techniques in use with at least one DMA. DMAs might be useful in supplementing traditional surveillance strategies with oncology drugs and other drugs with similar features. (i.e., drugs that may be approved on an accelerated basis, are known to have serious toxicity, are administered to patients with substantial and complicated comorbid illness, are not available to the general medical community, and may have a high frequency of "off-label" use).
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
INTRODUCTION: With increasing volumes of postmarketing safety surveillance data, data mining algorithms (DMAs) have been developed to search large spontaneous reporting system (SRS) databases for disproportional statistical dependencies between drugs and events. A crucial question is the proper deployment of such techniques within the universe of methods historically used for signal detection. One question of interest is comparative performance of algorithms based on simple forms of disproportionality analysis versus those incorporating Bayesian modelling. A potential benefit of Bayesian methods is a reduced volume of signals, including false-positive signals. OBJECTIVE: To compare performance of two well described DMAs (proportional reporting ratios [PRRs] and an empirical Bayesian algorithm known as multi-item gamma Poisson shrinker [MGPS]) using commonly recommended thresholds on a diverse data set of adverse events that triggered drug labelling changes. METHODS: PRRs and MGPS were retrospectively applied to a diverse sample of drug-event combinations (DECs) identified on a government Internet site for a 7-month period. Metrics for this comparative analysis included the number and proportion of these DECs that generated signals of disproportionate reporting with PRRs, MGPS, both or neither method, differential timing of signal generation between the two methods, and clinical nature of events that generated signals with only one, both or neither method. RESULTS: There were 136 relevant DECs that triggered safety-related labelling changes for 39 drugs during a 7-month period. PRRs generated a signal of disproportionate reporting with almost twice as many DECs as MGPS (77 vs 40). No DECs were flagged by MGPS only. PRRs highlighted DECs in advance of MGPS (1-15 years) and a label change (1-30 years). For 59 DECs, there was no signal with either DMA. DECs generating signals of disproportionate reporting with only PRRs were both medically serious and non-serious. DISCUSSION/CONCLUSION: In most instances in which a DEC generated a signal of disproportionate reporting with both DMAs (almost twice as many with PRRs), the signal was generated using PRRs in advance of MGPS. No medically important events were signalled only by MGPS. It is likely that the incremental utility of DMAs are highly situation-dependent. It is clear, however, that the volume of signals generated by itself is an inadequate criterion for comparison and that clinical nature of signalled events and differential timing of signals needs to be considered. Accepting commonly recommended threshold criteria for DMAs examined in this study as universal benchmarks for signal detection is not justified.