Stock Market Prediction using Machine Learning and VADER Sentiment Analysis of Twitter Data

Abstract

The stock market is one of the most significant impact factors for both domestic and global economies. Traders are always in search of somehow guessing the stock prices in the near future which can help them not just to increase their profit but to become top traders in the market. But the question is how to know the price ups and downs in a certain way? Thus, Stock Market Prediction has become an interesting research topic nowadays. Market movement is influenced by a number of variables, including company news and performance, social media, macroeconomic and micro-economic considerations, and overall industry performance, etc. The stock market is typically not that simple, linear, and smooth but the market trend can be determined by looking at the stock’s historical performance or its motivating elements. Due to the perspectives that investors and other social media users have regarding market movements, social media plays a key role in market analysis. In this research, we have proposed a generalized mechanism to predict stock prices using machine learning and sentiment analysis of Twitter data. We have used VADER which is a particularly attuned sentiment analysis tool for social media sentiments, and have overcome the challenge of considering slang words, emojis, emoticons, and punctuation towards sentiment analysis along with simple text to avoid information loss in the form of removing such real and visual indicators and to get rid of the data cleaning process. We have scrapped and used 10 years of Twitter data as well as stock prices of three different companies i.e. AAPL (Apple), TSLA (Tesla), and INTLC (Intel). Eight different classifiers like KNearest Neighbours(KNN), Logistic Regression(LogR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest(RF), Artificial Neural Networks (ANN), Long and Short Term Memory (LSTM), and 1 Dimensional Convolutional Neural Network(1DCNN) have been trained and evaluated. We have achieved an accuracy of 98% and DT is the top-performing model.