Text Classification Using NLP

Abstract

This research is based on the fine tuning of BERT model and applying long short term memory layers (LSTM). Bert which is already well known for text classification is being used along with other layers to further enhance the accuracy of text classification. Many existing Specific Bert models are available but they are only trained for a specific task. This paper shows the classification of four different classes: chats, emails, news and tweets. The method is pretty simple, at first dataset for each target class is collected and preprocessed using NLP libraries to remove extra and useless data from the datasets. The data-loader is prepared to feed the testing and validation data into a BERT base model. Before that, Bert tokenizer is used as Bert only takes data which is presented in a specific format having Special tokens ([CLS] and [SEP]). Using a recommended approach Bert is fine-tuned one by one for all target classes. The innovation is the introduction of LSTM layers merged with fully connected (FC) and some pooling layers in case. The trained output of Bert, which is Bert-embedding, is being used as an input to the LSTM model. Although, Bert alone could perform well, but for some complex datasets these additional layers have provided an edge. As LSTM layers are being used in bidirection to further capture the feature in the text to classify them more efficiently. Accuracy is enhanced overall. However, for binary classification with limited datasets there is a minor change in accuracy by introducing LSTM layers but for multi-classification with complex data, the accuracy is noticeable. For chats achieved accuracy is 99%; for emails, 98%; for news, 97%; and for tweets (complex data with multi-label sentiment analysis), 85%. These accuracies in comparison with alone Bert model are more efficient.