Real Time Object Detection And Scene Understanding for Blind

Abstract

Computer Vision is a field of artificial intelligence (AI) that enables computers and systems to derive meaningful information from digital images, videos, and other visual inputs. There is lot of work available in literature that is based on the manufacturing of visual aid to assist the blind or visually impaired people. This project involves computer vision as detect objects in the environment in real time and produce an audio output. It uses deep neural networks that are trained to detect objects in the environment hence making the project involve machine learning. A blind person entirely depends on someone for his daily life tasks and always wonders how the world is around him. So, for someone who does not even know what is present in his surrounding this project would help to at least try to understand his/her surroundings. A mobile application has been developed for this purpose which will identify objects in the surrounding and make the user understand about the surroundings through detected objects. The object detection model used is the Mobilenet SSD which is a Single Shot Detector model. YOLO was also tested for object detection but discarded due to very low FPS on mobile devices. TensorFlow Lite library is used for using object detection model on a mobile device. Android Text library is used for converting detected object output into voice. Open CV is used for preparing the object detection model and NumPy library has also been used for python programming part of the project. Some of the software’s used include Android Studio for building the mobile application, PyCharm IDE for designing, training, and testing the object detection model. Using all these tools the project can detect objects and describing the environment to the user on all mobile devices as it has minimal requirements like camera access, sound etc. which are already present in all smart devices.