Optical Music Recognition Using Deep Learning

Abstract

Optical Music Recognition (OMR) is an important task in music information retrieval from music sheet images. Precise detection of music symbols in a music sheet image is an essential component of any OMR system. The approaches that are currently being used do not show promising results on rare symbols. Additionally, an image of a music sheet contains a large number of densely packed symbols. This makes detection of smaller symbols using traditional approaches, which have been designed to detect fewer and larger objects in the images, difficult. In order to address these problems, a two-step approach for Music Symbol Detection has been presented. In the first step, the locations of the composite symbols has been identified instead of locating symbols primitives using a deep learning based approach inspired by state-of-the-art scene text detection method. In the second step, region based convolutional neural networks (R-CNN) has been used to detect symbol primitives in the image crops obtained from the localization step. The proposed approach targets the music sheets that contain a large number of music symbols. Additionally, primitive level symbol detection on image crops instead of full images allows us to sample rare symbols more often during the training process and helps to offset the effect of symbol imbalance in the data. The proposed approach has shown mAP of 24.9 and wmAP of 48.74 on DeepScores Dense Extended dataset [1]. Thus, this thesis makes twofold contribution through our work: (i) introducing a two-step music symbol detection algorithm and (ii) establishing baseline results with DeepScore Extended dataset.