Boosting Facial Recognition Capability for Faces Wearing Masks using Attention Augmented Residual Model with Quadruplet loss

Abstract

With the outbreak of COVID-19, people worldwide started wearing face masks to cover their mouths and noses to avoid the negative effects of the pandemic. The conven tional face biometrics systems were not designed to handle masked faces. Some facial features like the nose and mouth get hidden under the mask, resulting in performance degradation in face biometrics systems. Several studies also reported this degradation in face biometric systems performance when a mask is worn. In addition, several methods involving complex and computationally expensive deep learning models proposed for solving masked face recognition problems had downgraded performance in unmasked face recognition settings. Therefore, there was a need for a biometric face recognition system that could not only recognize faces with good performance in masked scenarios but has at least the same performance as state-of-the-art in unmasked scenarios. This thesis proposes the Masked Face Unveiling Model (MFUM) to cope with this problem. The MFUM works on top of existing face recognition models and is built on the concept that facial embeddings get corrupted for masked faces. This model makes masked facial embeddings of a person similar to unmasked facial embeddings of the same person and different from unmasked facial embeddings of other persons. Different ablation studies have been conducted using face recognition model backbones, MFUM architectures, and loss functions. Results are evaluated on the LFW dataset with synthetic masks and a real-world masked face recognition dataset, i.e., MFR2. We have reported the perfor mance of the proposed model in different unmasked and masked settings using evaluation metrics adopted globally for reporting the performance of biometrics systems, i.e., Equal Error Rate, False Match Rate, and Fisher discriminant ratio. The results show that the MFUM having Face Unveiling Attention Augmented Dense Residual Unit architecture and trained using Quadruplet loss has outperformed state-of-the-art methods in addition to other evaluated MFUM architectures with different losses.