Deep Learning Based Hybrid Approach for Shelves Monitoring and Planogram Compliance in Retails

Abstract

In retail management the continuous monitoring of shelves to keep track of the availability of the products and following proper layout are the two important factors that boosts the sales and improve customer’s level of satisfaction. As Artificial Intelligence (AI) is revolutionizing every sphere of life so enterprises are also focusing on using AI to reshape the ecology of retail industry. The studies conducted earlier are either performing shelf monitoring or verifying planogram compliance. As both the activities are important so this study presented a hybrid approach that deals with both of the activities. Our proposed approach consists of two modules; the first module detects fine grained retail products using one stage deep learning detector. For the detection part the comparison of three deep learning-based detectors (YOLO V4, YOLO V5 and YOLOR) is provided and the one giving the best result will be selected. The selected detector will perform detection of different categories of SKUs and racks. The second module perform planogram checking, for this purpose the company provided layout is first converted to JavaScript Object Notation (JSON Object) and then the matching is performed with the post processed retail images. The compliance reports will be generated at the end for indicating the level of compliance. The approach is tested both in quantitative and qualitative manner. The quantitative analysis demonstrate that proposed approach achieved an accuracy up to 99% on the provided dataset of retail, whereas the qualitative evaluation indicate increase in sales and customers’ satisfaction level.