Forensic Examination of handwritten scripts for writer identification

Abstract

Writer identification has always been a challenging task when it comes to defining a model that is robust to deliberate changes in handwriting styles. In our project, we introduced ‘WisNet’, a deep Convolutional Neural Network (CNN) based model for learning powerful representation of handwriting. This model was designed for forensically relevant writer identification tasks where a writer deliberately changes her/his handwriting style in multiple text samples (s)he writes. Our approach to this task involved 1) properly segmenting lines from paragraph documents 2) scanning patches of text and performing data augmentation to increase the number of training samples and finally 3) implementing a robust CNN for extracting powerful features from handwriting 4) implementing a loss function that allows the network to learn the differences and similarities between two writing samples. The model was then evaluated on publicly available ICDAR-2015 English handwriting dataset that consists of 165 pages of handwritten text from 55 writers (in three different handwriting styles per writer) in the training set and another 165 pages of handwritten text (yet another three different handwriting styles per writer) from the same 55 writers in the test set. We used the same data and evaluation protocol as were available in the ICDAR-2015 competition and the proposed system outperformed the state-of-the-art on the said forensic writer identification task with around 18% improvement. Furthermore, the proposed system also showed promising results on the publicly available HWDB1.1 data, for general writer identification task, which depicts the strength of WisNet. In extension of our work, we worked on the task of signature verification as well. We used the same model i.e. WisNet to learn the similarities and differences between original and forged signatures. We tested our model for signature verification task on the publicly available ICFHR-2010 signature dataset and were able to achieve promising results. A web application for this task was developed that tells us the difference between two signatures in real-time (it is effectively able to show a low difference value in a pair of genuine signatures and a high difference value in a pair of genuine and forged signature). The application can be deployed in banks to detect forged signatures in real-time with extreme precision.