Audio Based Advertisement Segmentation Algorithm

Abstract

Broadcast monitoring involves evaluating whether the correct advertisement was aired and for the correct length of time and at the time previously agreed upon. Advertisement forms the bulk of the revenue of television channels hence; advertisement monitoring and auditing becomes extremely important for the industry. Lack of e cient and inexpensive monitoring technologies directly impacts growth in advertisement revenues. Hence, there is a need for accurate and automatic computer based techniques for auditing broadcast content as the advertisers only pay when the television time allocated to the advertisement is veri ed. Heterogeneity in aired advertisements, signal quality, sizable amount of multimedia data and demand for high accuracy makes it a challenging problem to solve. Currently broadcast monitoring is performed manually i.e. using human effort, which is ine cient and prone to errors. Some computer based techniques have been proposed, however, these techniques lack whole industry coverage and, ability to scale to large number of broadcast channels and robustness. In this thesis we propose a solution for broadcast monitoring. Our solution uses an audio feature extraction based advertisement segmentation algorithm. When given an input stream (broadcast transmission) the algorithm matches it against a database of advertisements and automatically detects instances of aired samples within the input stream. Several di erent audio features were computed such as Spectral Entropy, Zero-Crossing Rate, Harmonic Ratio, Spectrum Basis, MFCC, GBFE, etc. The audio features were analysed using Average Dependency & Minimum-Maximum Distance criterion. The criterion judges the ability of an audio feature to di erentiate between classes of objects. The high quality features selected on the basis of ii iii the criteria are Gabor Filter Bank Feature (GBFE), Mel-frequency Cepstral Coe cient (MFCC) and MPEG7 Audio Flatness Mean. Subsequently, in order to increase scalability, robustness and recognition accuracy, dimensionality of the feature vectors is reduced using sequential forward oating feature selection guided by the Average Dependency and Minimum-Maximum Distance criterion. Experimental comparison of advertisement sequence classi cation was conducted to analyse the e ciency and e ectiveness of the feature extraction algorithms. Real-world dataset of 216 hours of television broadcasts (belonging to three separate channels) and 28 classes of advertisements were used in the experiment. While comparing di erent audio features it was observed that a scalable, robust and accurate television broadcast monitoring system can be designed using Gabor Filter Bank Feature (GBFE) which yields recognition accuracy of over 99.33%. Moreover, the results achieved are comparable with available literature in terms of accuracy and are superior in terms of speed, robustness and e ciency. It is also concluded that the feature selected along with the matching technique may be used to construct a scalable and e - cient system for television broadcast monitoring. However, there is room for improvement as in this work we target only audio analysis and video based matching would be required for a universal broadcast auditing system.