Semantic Scene Classification using AL with Camera for Autonomous Vehicles

Abstract

The fundamentals of an autonomous vehicle are perception of the environment, object detection, path planning, and control systems for vehicle actuation. There are two levels of perception, high-level scene classification and low-level depth perception. This thesis targets the high-level scene classification for an autonomous vehicle. High-level scene classification means to classify each pixel in a captured scene with an object class. With the advancement in technology and the rise of Artificial Intelligence, semantic segmentation models were developed performing this task. Hence among these models, Segformer was employed in this project for scene classification on a pixel level. A live video feed will be supplied to the model through a camera for classification. An IoT control has also been implemented to demonstrate the objective of providing a failsafe control switchover to a human in case of any complication. To classify the environment in real time so to be suitable for an autonomous vehicle, the entire AI algorithm was ported onto an embedded platform, the Jetson Nano, which is designed to manage the computational requirements of deep learning algorithms. The model was able to run at a mean of 1.71 fps on the Jetson Nano. The failsafe control was implemented using IoT over Wi-Fi on an ESP32.