An Unreal Engine Based Human Robot Interaction Framework

Abstract

Existing frameworks lack the support for testing Human Robot Interaction (HRI) research which in turn often has to be tested practically making it time consuming and expensive. To overcome this issue, a HRI framework based on Unreal Engine is proposed which consists of a virtual Nao or Pepper robot along with virtual humans with verbal and non-verbal behaviours in an environment. Machine Learning (ML) algorithms along with the behaviour of the virtual robot in response to the interaction with the virtual humans and the environment can be programmed using a Python API which communicates with Unreal Engine C++ in real time. Several experiments related to multiple aspects of HRI: (1) Verbal Interaction; (2) Non-Verbal Interaction; (3) Emotional Interaction were conducted in both the virtual and real world environments and the results were compared to validate the feasibility of the framework. A Reinforcement Learning (RL) algorithm was also tested to further indicate the usefulness of the framework. Through the use of a Virtual Reality (VR) headset, a human can be immersed in the framework to interact with the robot in real time.