RL-Based Controller of Fixed Wing UAVs for Tracking Moving Targets

Abstract

Unmanned Aerial Vehicles (UAVs) have witnessed a surge in applications across various domains, and while past research offers comprehensive material on conventional control, new applications require innovative control solutions. Deep Reinforcement Learning provides a promising framework for UAVs to autonomously learn control policies through interaction with their environment, thus mitigating the challenges posed by dynamic and uncertain operating conditions. This paper outlines the implementation of a reinforcement learning agent, using a Deep Deterministic Policy Gradients (DDPG) algorithm, to replace the conventional guidance and control system of a fixed-wing UAV that is designed to intercept a target. The 6-DOF model for this work was inspired by and derived from an open-source guided missile model, equipped with a gimballed radar tracker to detect target aircraft within the environment. An innovative reward shaping mechanism was used where the zero-crossings of the action were measured and fed back into the reward function to allow the learned actions to be significantly more stable. Proportional Navigation Guidance was used as the benchmark to evaluate the trained agent’s performance. The UAV adapted and optimized its guidance and control strategy highly effectively within the simulation environment, enabling it to intercept targets that conventional control failed to capture. This research can be used to pave the way for gimballed platforms to be used for radar seekers and cameras for commercial UAVs for tasks such as tracking, following, intercepting, etc.