Identifying Human Emotions from Single Walk Step Using a Body-Mounted Inertial Sensor

Abstract

Any conventional gait pattern comprises of two types of feature, one is temporal which shows variation over time and the other is non temporal processed feature set which remains constant over time. To a considerable extent, non temporal feature set of time series sequence has been disregarded in many deep learning research. Despite it being a crucial component of time series data, existing researchers mostly use temporal sequence to classify and categorize time series sequence. Thus this research proposed the idea to train a model for both set of features of step gait data, collected by IMU sensors, for emotion classification. We trained two deep learning architectures for six basic emotions i.e. Happy, Sad, Anger, Fear, Disgust and Surprise. The first proposed architecture, DensEmoNet, is fully convolutional neural network trained for complete feature set of the step gait data. The temporal feature based architecture, on the other hand, is built with the intention of training both feature set separately. Complete temporal feature set is used to hold the structural pattern of the signal, but feature categorization is employed to identify the best processed feature combination for classifying emotions, which will subsequently be applied to each temporal feature based training architecture. In this work, the gait sequence, which includes 3D accelerometric and 3D gyroscopic data, is employed as a temporal feature. Moreover processed characteristics are either acquired during the data collection phase or derived by using temporal features. In temporal feature based architecture, processed features are trained with Conv layers, similar to the DensEmoNet, and temporal features are trained with RNN variants. On a combination of the binary, tertiary, quarter, penta, and hexa classes, accuracy scores are obtained. The categorization of emotions into hot and cold classes is also used to evaluate the architecture’s reliability. In all combinations of emotion classification, temporal feature based Architecture outperforms DensEmoNet and provides an accuracy of 87.71 for the categorization of six emotions while maintaining the accuracy span of 87 xi to 98%. DensEmoNet on the other hand gives stable accuracy within the range of 80 to 97 for all of the classes. With such a high accuracy score, it is clear that the suggested approach will make a valuable addition to the field of study.