Performance comparison of Machine Learning approaches for detection of ships in satellite imagery

Abstract

Remote satellite sensing has a vast potential for Maritime Domain Awareness (MDA). The challenges at sea are unlimited, however, a few important aspects include security,

safety, environment, weather and disaster response (e.g. for tsunami). These chal- lenges can be managed efficiently if accurate MDA is available to respective response

authorities. An effective method to achieve MDA is using optical satellites for detec- tion, classification, and identification of ships. This has been possible due to increase

in number of optical satellites and open source availability of satellite imagery data. This research is aimed to achieve ship detection using various Machine Learning (ML) approaches. A maiden dataset of satellite images has been prepared using available open-source satellite imagery. A comprehensive Exploratory Data Analysis (EDA) for this dataset was undertaken to effectively utilise the parameters in this research work. This dataset can drive future research for Maritime Domain in fast evolving AI development environment. Eight models were developed and trained using the developed dataset. Initially, HOG based feature extractor with 03 x ML algorithms which include Decision Tree, Random Forest and Support Vector Machine (SVM). The maximum accuracy achieved by these models were 88% for Random Forest, 70% for

Decision Tree and 99% for SVM. Secondly, 04 x Regional Convolutional Neural Net- work (RCNN) based classification model were developed and trained to achieve ship

detection. The RCNN based models include Baseline Multilayer Perceptron (MLP), LeNet, VGG 16 (Visual Geometry Group) and VGG 19. The RCNN models achieved the highest accuracy of 85% for MLP, 97% for LeNet, 96% for both VGG 16 and VGG 19.The Last trained model is based on higher level deep learning U-Net architecture. U-Net achieved accuracy of 97%. Finally, a performance matrix was developed which

v

can act as a guide to select suitable algorithms for specific application or a task in

future. The comparison shows that MLP, LeNet, VGG16/19 and U-Net are compu- tationally intensive. Random Forest, SVM and Decision Tree, on the other hand, are

less computationally intensive. Considering the state of the art image classification algorithms like Hierarchical Design (HieD), PSMEWT (Phase Saliency Map Extended Wavelet Transform), Yolo, DenseNet and SVDNet the accuracy achieved by U-Net, SVM, VGG 16 and VGG 19 in this research work are comparable and in some cases better.