APOLLO: A Multi-Modal Framework for a Socially Evocative Robotic System Based on Human-Human Interaction

Abstract

This manuscript presents a novel modular layer architecture to make human robot interaction more natural and socially acceptable. The proposed framework employs Haar cascade features to classify the human subject for an emotional state using a Gabor filter bank which uses both real and complex parts of the Gabor Kernel. The presented framework harnesses the effects of nonverbal modality of facial expressions during a human robot interaction to make the whole process more natural. Extended Cohn- Kanade (CK+) Facial Expression database is used to train a Support Vector Machine (SVM) classifier based on posed frontal faces. Along with multiple affine transformations, pre-processing of the Facial Action Coding System (FACS) coded CK+ images is done using Haar cascade face and eye detectors. Gabor Filter Bank is implemented for feature extraction from the processed images of CK+ database. The generic Gabor kernel is modified to use both the real and imaginary part of the convoluted signal. SVM are then trained on extracted features to prepare a classifier for any given image. The proposed architecture is designed to classify six of human emotions of happiness, sadness, fear, anger, disgust and surprise, which are established as the primary human emotions. Behavioral analysis and designing for the action cycle was done using a series of surveys. The whole framework is then deployed using Aldebaran’s NAO Humanoid robot as a proof of concept platform. The modular design helps in custom fitting the framework to several different type of systems. Each of the building block i.e. Perception, Cognition and Action is necessarily independent which give the user the ability to tweak each of the underlying processes according to requirement by a certain system.