Focus and Engagement Level Detection Using Computer Vision and Machine Learning in a Classroom Environment

Abstract

Due to Covid 19, the global education system has changed toward online learning, which has a high dropout rate. Therefore, it is vital that students maintain their level of interest. Therefore, detection of engagement level alone is insufficient for analyzing and improving learning and teaching techniques. To promote student engagement in STEM and online learning environments, technologies such as AR/VR and Haptics should be implemented. Utilizing facial emotion, body pose, and head rotation, a web-based computer vision system is developed and implemented to identify student involvement levels using webcams during tasks such as online classrooms, haptic interaction, and augmented reality. In addition, an AR and Haptics-based World Map is being designed and developed. To evaluate and compare three types of learning scenarios, namely (1) Traditional, (2) Augmented Reality-based, and (3) Haptics-based, two methods are employed: (1) Trained Computer Vision models are tested for 3 scenarios, and (2) A user study is conducted using the Positive and Negative Affect Schedule (PANAS) Questionnaire and NASA-Task Load Index, from which conclusions are drawn. The results of a comparison of Traditional, Augmented reality, and Haptics-based learning indicate that Haptics and Augmented Reality-based learning are the most immersive and increase levels of engagement during online learning and STEM training, whereas Traditional learning methods are the least effective during online classes. User studies and computer vision models are utilized to validate the results.