3D Lidar Vision 3D Lidar Point Cloud Based Vehicle Detection And Classification

Abstract

We adopted the methodology of Point Fusion for combining Image Data and 3D Lidar Data using sensor fusion through a neural network. The procedure involves passing image data through a 2D object detector which in our case was Faster R-CNN (pretrained on MS COCO), then using the region of interests obtained as 2D Labels predicted by 2D object detector to crop point clouds and obtaining its regions of interest. The cropped images were passed through a ResNet Model (pretrained on ImageNet) and final average feature layer was extracted from it. The cropped point clouds and the obtained features were passed through a neural network that contained PointNet and Point Fusion network layers. The results were obtained in the form of 8 corners for each bounding box for each vehicle/pedestrian present in the image