Intelligent Framework for Automated Failure Prediction and Classification of Mission-Critical Autonomous Flights and Autopilot Software

Abstract

Autonomous flights are the major industry contributors toward next-generation developments in pervasive and ubiquitous computing. Aerial vehicles are designed to perceive their surrounding environment through various sensors. Contextual infor mation from the measured parameters is then fed to the primary autopilot software. Commanded instructions by the control logic are accordingly issued to the respec tive actuators, which regulate servo motors for adjusting the speed and direction of the aerial vehicle through control surfaces and rotary wings. This way it manages the flight operations, from takeoff to landing through flying between waypoints, and interacts with the outside world by implementing the flight dynamics model. Due to physical exposure, real-time interaction with the actual atmosphere, and the data driven flying nature of aerial vehicles; there exist high risks of physical impairments and cyber exploitation. If not anticipated and then timely controlled, failures occur ring during the flight can lead to unsafe flight conditions and may result in severe consequences. When control logic handling the fault doesn’t anticipate the context of the issue and autopilot software continues executing flight operations without necessary adjustments; the problem’s dissemination to the critical parts can disrupt the flight mission and may even result in mid-air collision or the ultimate crash of the flying machine. Various solutions have been formulated to mitigate aerial vehicle parts, systems, and software failures while ensuring their safety and reliability. We performed an ex tensive literature review to explore the evolution of these developments and classified them into logical categories by progressive citing of the related work. Establishing connectivity of the literature to various reported flight accidents and aerial vehicle crashes; we identified notable research gaps in our problem domain and presented our research contributions to address them. We recognized that the humongous amount of sensory data being generated throughout mission-critical autonomous flights makes them an ideal candidate for applying advanced data-driven machine learning techniques to identify intelligent insights related to failures for instant re covery from emergencies. In this thesis, we proposed deep learning based failure prediction and classification frameworks for in-flight data-driven vulnerabilities of autonomous aerial vehicles. We have provided solutions for two types of failures; The first is physical impairments which include in-flight engine fire and control surface failures. The second is cyber exploitations which include incorrect percep tion of the surrounding environment either because of a compromised sensor block or modified sensory data due to counterfeit wireless signals. We proposed: i. A novel machine learning based theoretical framework for real-time monitoring and failure analysis of mission-critical autopilot software: It has been introduced to protect the autopilot software from run-time data-driven cyber exploits that may be caused by missing, jammed, or spoofed data values; due to malicious online cyber activities. The effectiveness of the proposed framework has been demonstrated by the evaluation of a real-world incident of grounding an aerial vehicle by the actors in their vicinity without the intent of the original equipment manufacturer (OEM). The results showed that the reported undesired but successful cyber exploit may have been avoided by the effective utilization of our proposed cyber defense approach, which is targeted at software failure prediction, detection, and correction for autonomous aerial vehicles. ii. An LSTM-based novel framework for failure prediction, detec tion, and classification of mission-critical autonomous flights: The pro posed framework utilizes long short-term memory (LSTM) recurrent neural network (RNN) architecture to analyze time series data and has been applied to the AirLab Failure and Anomaly (ALFA) dataset, which is a comprehensive publicly available benchmark database of various fault types in a fixed-wing unmanned aerial vehicle. The proposed framework is capable to predict engine and control surface failures with an average accuracy of 93% and the mean time-to-predict a failure is 19 sec onds before the actual occurrence of the failure, which is 10 seconds better than the current state-of-the-art. Failure detection accuracy is 100% and the average detec tion time is 0.74 seconds after happening of the failure, which is 1.28 seconds better than the current state-of-the-art. Failure classification accuracy of the proposed framework is 100%. iii. A first-ever comprehensive VTOL AAV (short-form of Verti cal Takeoff and Landing Autonomous Aerial Vehicles) sensors failures dataset: It has been generated to be published as a public repository containing 70 autonomous flight data with more than 7 hours of flight time; comprising 3+ hours of (each) pre-failure and post-failure data. Our Biomisa Arducopter Sensory Critique (BASiC) dataset includes raw along with serialized, interpolated, and smoothed data with scenarios for six different types of sensors failures, including global positioning system (GPS), remote control, accelerometer, gyroscope, compass, and barometer. iv. A transformer-based novel framework for failure prediction and classification of mission-critical autonomous flights: The proposed frame work utilizes state-of-the-art transformer architecture to analyze our multivariate spatio-temporal data and has been applied to the processed part of the BASiC dataset. We have also performed various ablation studies i.e., evaluation of trans former performance for failure analysis with respect to different hyperparameter tuning, and presented the results to demonstrate their change effects. The proposed framework is capable to predict sensor failures with an average accuracy of 83% and the mean time-to-predict a failure is 1 minute and 2 seconds earlier than the actual failure, which is 30 seconds quicker than the LSTM-based failure prediction approach. It can also predict engine and control surface failures of the ALFA dataset with 100% accuracy and the mean time-to-predict the failures is 34 seconds, which is respectively 15 seconds and 25 seconds quicker than our LSTM-based approach and state-of-the-art from literature. Failure classification accuracy of the proposed framework is 100%. The performance analysis shows the strength of the proposed methodologies to be used as real-time failure prediction and classification frameworks for eventual deployment with actual mission-critical autonomous flights.