Fixed-Value MBLL based Cognitive Hemodynamic response assessment using P-fNIRS system: Applications to Deep Learning Brain Machine Interface (BMI)

Abstract

Humans in the modern systems not only interact with other humans but also have to interact with intelligent machines, robots in form of cyber physical systems to collaborate in order to carry out different tasks in real working environment. The modern Industrial system comprises of humans, machines, and cyber systems with a collective aim of optimized industrial manufacturing objectives, human factors, and ergonomics goals. Different macro-human factors are considered while designing and formulating human work safety of such systems and one of the important neuroergonomic factors in is the Cognitive and Mental Workload (C-MWL). The mental workload (MWL) in the human’s brain is measured with difference non-invasive neuroimaging techniques. Most of the cognitive load measuring methods either require massive system protocols like fMRI (functional magnetic resonance imagining), positron-emission tomography (PET) or strict human anatomical movements restrictions like electroencephalogram (EEG) and magnetoencephalography (MEG). To address these limitations, fNIRS (functional Near infrared Spectroscopy) is used in this research to measure the hemodynamic changes in the human brain’s tissues as a measure of the brain activity. The brain’s hemodynamic signals are measured using a light weight portable fNIRS system (P-fNIRSSyst) that is designed to measure relative change in concentration of chromophores (oxy and deoxy hemoglobin) in brain tissues. In this study a novel variant of MBLL (Modified Beer-lambert Law) is designed by keeping the previous intensity value as a reference by taking the average from initial four seconds activity stimuli in optical density. The four second stimuli average in novel and important in calculation the changes in concentration of chromophores. This novel variant of MBLL is Fixed Value - Modified Beer-lambert Law (FV-MBLL). In this research, MWL is measured and classified in different real time working environments. The two-state cognitive load is measured with fNIRS system and classified using FV-MBLL using machine learning techniques like k-nearest neighbors (k-NN), support vector machines (SVM), and artificial neural networks (ANN). The classification accuracies of FV-MBLL are better than MBLL. The research further explores the classification capabilities of deep neural networks (DNN) such as convolutional neural network s (CNN) and Long short-term memory (LSTM) for the first time in assessment and classification of four- iv state MWL. The classification accuracies of LSTM outperform not only ML algorithms (SVM, KNN and ANN) but CNN as well in classification of multi-state MWL. The research experimental validation is performed using the accuracies with MWL that are further utilized in neurorehabilitation as neurofeedback to operate bionic systems using Brain Machine Interface (BMI).