Shadow Removal using Attention-based GANs

Abstract

Shadows are natural artifacts present in images that can hinder various Computer Vi sion tasks such as object detection, tracking, segmentation, and scene analysis. This research introduces an innovative approach to detect and remove shadows from single RGB images using Attention-based Generative Adversarial Networks (GANs). The pro posed methodology employs a deep-learning model comprising Attention-based GANs, featuring two generators and two discriminators, to effectively identify and eliminate shadows. Subsequently, the shadow-free image generated by the GANs undergoes a post-processing step to refine shadow regions using a shadow mask. This post-processing stage combines traditional image processing techniques, including histogram matching, custom filters, and shadow boundary detection and estimation, to enhance the accu racy of shadow removal. Additionally, we used a large-scale benchmark dataset named "Extended ISTD," consisting of 5352 triplet images (shadow, shadow mask, shadow-free samples), facilitating both shadow detection and removal tasks. This dataset encom passes a diverse range of dark and hard shadow images, as well as multi-color contrast shadow images, serving as an extended version of the publicly available "ISTD Dataset." Upon training the Attention-based GANs on the provided dataset and applying the proposed post-processing step, an RMSE of 5.28 is achieved. The proposed methodol ogy demonstrates efficient shadow removal capabilities, even in scenarios involving dark, hard shadows, and multi-color contrast shadow images.