Natural Language Descriptions of Images

Abstract

Auto image captioning has recently gained a lot of attention especially because of the use of social media. We are in an era where images have become rather important. Few years ago this was a far fetched idea that a computer can understand a scene and describe it without any human assistance but with the advancement in machine learning , computer vision and natural language processing, a way has been paved for the researchers. Because of this researchers are putting in a lot of effort in this domain and are trying their best to innovate and develop tools which would help in describing the images for the users. A lot of work has already been done in this domain and a lot is still being done. Google and Microsoft are investing heavily in the field. Many models have been developed which can caption an image with great accuracy. There are models which can describe different objects in the images, there are models which can describe different regions , and there are models which can describe a whole image in a sentence. We have worked with the combination of all of the above. Our model describes every region in the image and then creates a meaningful sentence out of those region descriptions.This has helped us in creating dense captions for images, thus providing more information Our Project is Natural Language Descriptions of Images - SceneCap , which has the above mentioned model integrated inside an app, which can be used to take real-time pictures of the scenes to describe them with great detail and understanding. The application allows it to be portable and accessible from any smartphone. This allows our model to be flexible and readily available in the palm of anyone’s hands.