Single Camera-Based Abnormal Activity Analysis for Surveillance Applications

Abstract

The project is to develop a framework for abnormal activity detection. This includes the collection of datasets, which were collected personally as well as from Internet platforms such as Twitter and YouTube. The datasets included different scenarios such as wall climbing, entrance into sensitive premises and intrusion through a window. For object detection the YOLO algorithm is used and only “person” class is detected. The algorithms provided us with accurate bounding boxes of detected objects and this data is further used to track people and analyze their activity. For object tracking the Deep SORT algorithm is used which retains the identities of detected objects in consecutive frames and helps to extract trajectories of detected person class. Trajectories are then analyzed to detect abnormal activity. The intrusion is considered an abnormal activity for this project. For greater accuracy of intrusion detection some features were introduced, which includes the translation of centroid to the bottom of the detected bounding box to avoid any intrusion remain undetected, since in majority of cases the area of restriction is highlighted on bottom surfaces. The distance between centroid and line of restricted area is calculated by using the formula of the distance between a point and line of restriction. This made the algorithm robust enough to detect intrusion from any side of the restricted zone. The packages required by the algorithm were installed and dependencies were resolved for the proper execution. The evaluation was performed on collected datasets as well as on the real time video feed and the algorithm was able to detect intrusion in all scenarios. The developed algorithm is then ported to an edge device Jetson Nano for real-time analyses. For optimization of algorithm different techniques were used such as down-sampling and the usage of light weight model. The input video feed was given by a single camera and after analyzing the input video Jetson Nano generates a warning if it detects an intrusion. For complete setup all the input and output devices such as power adapter, internet connection, display were managed for the smooth operation of project.