An Intelligent Hand Gesture Recognition System

Abstract

Gesture and emotion recognition plays an important role in the judgement of human behavior. The field of engineering has adopted this science in recent years for development of intelligent and companion systems. Various gesture recognition software aid in medical domain for betterment of human health and general understanding. This project focuses on the development of a glove for mapping hand gestures and emotions. Four different gestures i.e. thumbs up, pointing finger, fist open and fist close are used. The emotions involved are sad, happy and angry. Emotions are detected by viewing variations in the values of jerk, velocity and acceleration. For this interfacing of flex sensors with myRio for data set collection has been done. A questionnaire has been developed based on five different models including well known categorical as well as dimensional (2D,3D) models to perceive the emotions of observer sitting in front and therefore validating the emotional state based on chosen parameters. The models used were: Paul Ekman’s model, Russell’s circumplex model of affect, PANAS model, PAD model and Plutchik's Model of Emotions. In this study, a relationship is developed between the motion of the hand and the perceived emotion. The validity of the perceived emotion by the user is later checked using five different emotional models. The motion characteristics such as velocity and acceleration are changed systematically to observe the change in the perception of affect caused by the robotic motion. The perceived affect is then marked by the user on all three emotional behaviour models. The novelty of the research lies in two facts: First comparative study of various emotional models used for measuring perceived emotions using Flex sensors. From the results produced it can be concluded that the emotions perceived by the user is the same on all the five scales, validating the reliability of all the five emotional scale models and also that by changing the motion characteristics parameters i.e. velocity, jerk and acceleration, the emotions perceived by the user also changes.