Dimensionality Reduction: A Comparison Between Principal Component Analysis and Deep Learning Approaches

Abstract

The emergence of big data has increased the importance of dimensionality reduction in every field related to data analysis. The search for an optimal and generalized dimensionality reduction method is very important to reduce and optimize the time needed for data execution. Dimensionality reduction is the most needed part in any type of data pre-processing and visualization. The idea of this research study is basically to compare deep learning methods (Auto-encoders) with principal component analysis (PCA) for dimensionality reduction and to investigate about the performance of these methods based on different datasets. Due to the lack of any general evaluation measure for dimensionality reduction methods, we have tried to compare the methods based on the measure of classification accuracy and the execution time of the models. For this purpose, a baseline classification model has been developed for each dataset using logistic regression by utilizing all the original information of the data. The baseline accuracy is then used as an evaluation measure for comparing the methods performance. The methods have been applied on two different types of real-world datasets: Human activity recognition system (HAR) dataset and MNIST dataset. The results show that PCA performs better both in terms of the classification accuracy and execution time on the HAR. Deep auto-encoder improved the classification accuracy of the logistic model for MNIST data, but it was not able to optimize the time execution of the model. The performance of auto-encoders in terms of the classification accuracy was comparable but their training time was longer. In future as more advancements come in the field of deep learning for optimizing the algorithms execution time, it may outperform other dimensionality reduction methods.