Journal
DRUG SAFETY
Volume 44, Issue 1, Pages -Publisher
ADIS INT LTD
DOI: 10.1007/s40264-020-00996-3
Keywords
-
Funding
- US FDA
Ask authors/readers for more resources
ADE Eval evaluated a range of NLP techniques for identifying ADEs mentioned in publicly available FDA-approved drug labels. It aimed to reflect pharmacovigilance practices within the FDA and model possible pharmacovigilance use cases.
Introduction The US FDA is interested in a tool that would enable pharmacovigilance safety evaluators to automate the identification of adverse drug events (ADEs) mentioned in FDA prescribing information. The MITRE Corporation (MITRE) and the FDA organized a shared task-Adverse Drug Event Evaluation (ADE Eval)-to determine whether the performance of algorithms currently used for natural language processing (NLP) might be good enough for real-world use. Objective ADE Eval was conducted to evaluate a range of NLP techniques for identifying ADEs mentioned in publicly available FDA-approved drug labels (package inserts). It was designed specifically to reflect pharmacovigilance practices within the FDA and model possible pharmacovigilance use cases. Methods Pharmacovigilance-specific annotation guidelines and annotated corpora were created. Two metrics modeled the experiences of FDA safety evaluators: one measured the ability of an algorithm to identify correct Medical Dictionary for Regulatory Activities (MedDRA (R)) terms for the text from the annotated corpora, and the other assessed the quality of evidence extracted from the corpora to support the selected MedDRA (R) term by measuring the portion of annotated text an algorithm correctly identified. A third metric assessed the cost of correcting system output for subsequent training (averaged, weighted F1-measure for mention finding). Results In total, 13 teams submitted 23 runs: the top MedDRA (R) coding F1-measure was 0.79, the top quality score was 0.96, and the top mention-finding F1-measure was 0.89. Conclusion While NLP techniques do not perform at levels that would allow them to be used without intervention, it is now worthwhile exploring making NLP outputs available in human pharmacovigilance workflows.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available