V-CAS: A Realtime Vehicle Collision Avoidance System Using Deep Learning on Multiple Camera Streams

Abstract

This research introduces a robust real-time Vehicle Collision Avoidance System (V-CAS) aimed at enhancing vehicle safety through environmental perception-based adaptive braking. V-CAS utilizes the advanced vision-based transformer model RT-DETR, DeepSORT tracking, speed estimation, brake light detection, and an adaptive braking mechanism. It computes a composite collision risk score from vehicles’ relative accelerations, distances, and detected braking actions, leveraging brake light signals and trajectory data through multiple camera streams for improved scene perception. Implemented on the Jetson Orin Nano, V-CAS enables real-time collision risk assessment and proactive mitigation via adaptive braking. A comprehensive training process was conducted on various datasets for comparative analysis, followed by fine-tuning the selected object detection model using transfer learning. The system’s effectiveness was rigorously evaluated on the Car Crash Dataset (CCD) from YouTube and through real-time experiments, achieving over 98% accuracy with an average proactive alert time of 1.13 seconds. Results show significant improvements in object detection and tracking, enhancing collision avoidance compared to traditional single-camera methods. This research highlights the potential of low cost, multi-camera embedded vision transformer systems to advance automotive safety through enhanced environmental perception and proactive collision avoidance mechanisms.