Pakistan Stock Market Prediction with Sentiment Analysis and Influence of Twitter User

Abstract

The stock market fluctuates a lot on daily basis and these shifts can be difficult to pre dict. Stocks prediction is a major challenge as it depends on many factors. Different events show that the influence of a user on social media, who tweets about a company, can impact the stock market movement in terms of high and low prices. In this research, the main objective is to explore the influence of Twitter users along with their tweet sentiment values to predict the next day’s stock opening price of Pakistan’s petroleum companies. Two types of the dataset were used in this study. The one was collected from Twitter in which tweets were extracted related to Pakistan’s petroleum companies along with user’s profile-based attributes and tweet-based attributes. The second dataset of stock historical prices is scraped from Pakistan Stock Exchange (PSX) website. Data preprocessing is performed on both datasets involving data cleaning, data normaliza tion, and imputation. Valence Aware Dictionary and sEntiment Reasoner (VADER) was used for calculating the sentiment score of users’ tweets. For influence detection, this research study introduced a comprehensive and composite approach to identify the overall influence of Twitter users among other users. The final dataset is then used with different machine learning algorithms, ensemble methods, and neural network models for the prediction of the next day’s stock open price. After the selection of hyper pa rameters, the models were trained using a training dataset. Trained models were tested using a test dataset and the performance of each model is observed by using standard evaluation indicators such as Root Mean Squared Error (RMSE) and R-Squared (R2). XGBoost ensemble model achieved the finest prediction results by achieving the lowest Root Mean Squared error of 0.026 and highest R-squared of 0.98. The final results demonstrate that the proposed idea of stock prediction with the help of the influence of Twitter users along with their sentiments could be used to predict the next day’s stock opening price, which can help investors to make informed decisions.