Immersive VR based Digital Twinning for Mobile Robotic Platforms

Abstract

This work deliberates building an intelligent and intuitive telepresence human robot interaction framework that liberates the human from handling technical difficulties and increases the resourcefulness of remote service robot. The complementing framework comes underutilization for tasks such as inspection, exploration, and feature collection from unknown, dynamic, and stochastic environments. Comparative to existing works it provides multimodal control and seeks user friendly features, realism and immersivity via a VR android application. The framework allows the user to switch the mode of operation from direct control to a collaborative/semi-autonomous one with the latter being controlled by an actor-critic network for autonomous local navigation. The user experience is also evaluated with real time experimentation and monitoring of human performance using BCI and analyzing results accumulated from the presence, usability, and sustainability questionnaires filled in by test subjects. Mean presence and usability scores of 97.6% and 94.71% supports the claim of this framework being immersive, intuitive, and collaborative with a simultaneous reinforcement from the Human performance metrics evaluation during the activity