Efficient Feature Selection for Machine Learning Using Biologically Inspired Algorithms

Abstract

Big data has become a ubiquitous feature of modern society, providing opportunities for extracting valuable insights from large datasets. However, the sheer volume, variety, and complexity of big data make it difficult to process and analyze, particularly in the field of Machine Learning (ML). Therefore the technique of dimensionality reduction is of paramount importance in ML, data mining and information retrieval domains. Fea- ture selection is a crucial step in the preprocessing of data for machine learning modèle for dimensionality reduction, as it involves selecfing a subset of relevant features to improve the model’s accuracy and reduce the risk of overfitting. Metaheuristic algo- rithm based wrapper methods are a type of feature selection technique that involves using the predictive performance of a learning algorithm to create a reduced feature set. The traditional methodology involves the use of K-Nearest Neighbour based accuracy maximization as the colt function, which is seen to produce sub-optimal solutions in large sample spaces and is relatively resource intensive. In this work, a novel meta- heuristic algorithm, namely the Hybrid Sine Cosine Firehawk Algorithm (HSCFHA) is proposed. The proposed feature selection technique uses this hybrid algorithm and aime to eliminate insignificant and redundant features by including the minimisation of dataset variance in the cost function. Furthermore, the hybrid algorithm uses the best features of the combined algorithm to improve the exploration ability. The proposed technique is tested on 22 various University of California Irvine datasets containing low, medium and high dimensional datasets and compared to the fraditional KNN based approach. Moreover, the technique is also compared with other state-of-the-art meta heuristic techniques, namely Parficle Swarm Optimizer, Grey Wolf Optimizer, Whale Optimization Algorithm, Hybrid Ant Colony Optimizer and Improved Binary Bat Al- gorithm. The results show significant improvements over previous techniques in terms of minimal loss in essential data while reducing the size of the raw data in considerably less time as well as a well-balanced confusion matrix.

Supervisor: Dr. Ahsan Saadat