Adverse Drug Reaction (ADRs) Extraction Using Transfer of Learning

Abstract

Adverse Drug Reactions (ADRs) are significantly harmful for health. Existing studies utilize traditional and deep learning techniques to detect ADRs from the given text. Bidirectional Encoder Representations from Transformers (BERT) overcame the predominant neural networks bringing remarkable performance gains. However, training BERT is computationally expensive which limits determining the most important hyper parameters for the downstream task. Furthermore, developing an end-to-end ADR extraction system comprising two downstream tasks i.e. text classification for filtering text containing ADRs and extracting ADR mentions from the classified text is also challenging. In this work, we present an end-to-end system for modelling ADR detection from the given text by ne-tuning BERT with a highly modular Framework for Adapting Representation Models (FARM). FARM provides support for multi-task learning by combining multiple prediction heads which makes training of the end-to-end systems easier and computationally faster. In the proposed model, one prediction head is used for text classification and another is used for ADR sequence labelling. The model is fine-tuned on the data collected from Twitter and PubMed abstracts. The proposed model is compared with the state-of-the-art techniques and it is shown that it yields better results for the given task.