Leveraging Deep Neural Networks and Surface Electromyography for Real-Time Airwriting Gesture Recognition

Abstract

Airwriting enables users to write letters or characters in free space using hand or finger movements, with potential applications in human-computer interaction, virtual reality, augmented reality and development of assistive technologies. Despite advancements in gesture recognition technology, dynamic airwriting faces challenges with accuracy and often lacks real-time capabilities, limiting its application in non-verbal communication and rehabilitation devices. The primary objective of this research is to develop a novel real-time deep learning-based framework for airwriting recognition using surface electromyography (sEMG). This study presents a technique for real-time identification of uppercase English language alphabets written in free space by analyzing the electrical activity of forearm muscles involved in writing letters. The proposed framework involves sEMG data collection from 16 right handed healthy subjects with no neuromuscular or motor impairments, signal preprocessing, feature extraction, classification using Convolution neural network (CNN), Deep neural network (DNN) and Recurrent Neural Network (RNN).The best performing model was implemented in real-time and it was evaluated using performance metrics such as accuracy, precision, recall, F1 score, Confusion metrics and latency. Results show that 1 Dimensional (1D) CNN outperforms other models (p<0.05) and achieved an offline test accuracy of 89.81 ±0.87% and an average real-time test accuracy of 73.71 ±8.46% across subjects. The individual model of each subject performed even better, with an accuracy of 90.01 ±2.85% on offline testing of data and 75.45 ±1.53 % in real-time alphabet prediction. Thus, this work highlights the potential of deep learning models for real-time airwriting detection and provides foundations for sEMG-based airwriting applications in healthcare and telemedicine.