Children’s Spontaneous Facial Expression Recognition Using Deep Learning Methods

Abstract

For a child to incorporate himself into the society as a productive individual, it is vital that his emotional development is sound. In recent years, the emergence of several real-world applications has shifted the focus of technology towards enabling machines to decode and understand the emotional signals of human beings. However, in research, very often emotion recognition is limited to adults, not considering the fact that children can develop an awareness of various emotions at a very early stage, thus, the ability of machines, to distinguish various facial expressions of children, still needs to be explored. In the absence of a standardized database, it yet remains a challenge. Thus a set of benchmarks are required to establish a standardized comparison. In this paper, a system based on convolutional neural networks has been proposed to automatically recognize children‟s expressions. For this purpose, a video dataset for Children‟s Spontaneous facial Expressions (LIRIS-CSE) has been used. In our proposed system, pre-trained Convolutional Neural Network (CNN) such as VGG19, VGG16, and Resnet50 have been deployed as feature extractors and then models such as Support Vector Machine (SVM) and Decision Tree (DT) have been used for classification. We have tested their strength with various experimental setups such as 80-20% split, K-Fold Cross Validation (KFold CV), and Leave One out Cross-Validation (LOOCV), for both image-based and videobased classification approaches. Our research has achieved a promising classification accuracy of 99% for image-based classification via features of all three networks with SVM using 80-20% split and K-Fold CV. Video-based classification results have been reported as well, where we have managed to achieve 94% accuracy via features of VGG19 with SVM using LOOCV. Our achieved results are better as compared to the original work, where they have achieved an average image-based classification accuracy of 75% on their designed LIRIS-CSE dataset.