Abstract:
Machine learning algorithms have been widely used in various applications and areas.
Hyper-parameters of machine learning models must be optimized in order to fit them into various problems. Performance of machine learning models is highly dependent on the selection of the best hyper parameter configuration. It often requires deep knowledge of machine learning algorithms and appropriate hyper-parameter optimization techniques. Although several automatic optimization techniques exist, they have different strengths and drawbacks when applied to different types of problems.
In this research, we have provided a comprehensive analysis of several state-of-the-art
hyper-parameter optimization techniques including their strengths and limitations, and
discussed how to apply them to various machine learning and deep learning models.
Also, on the basis of extensive literature review, a taxonomy for the hyper-parameters
optimization techniques is proposed. Moreover, experiment is conducted on benchmark
dataset to compare the performance of different optimization methods. This research
will help industrial users, data analysts, and researchers to better develop machine learning or deep learning models by identifying the proper hyper-parameter configurations effectively.
