Transformer based approach for inappropriate content detection in Urdu

Abstract

The popularity of social networking sites and online forums has increased the spread of harmful and improper content. While many research have looked into this issue in various languages, there is a big void in the literature when it comes to employing deep learning techniques in native Urdu language. This research is a continuation of Atten tion based Bidirectional GRU hybrid model for inappropriate content detection in Urdu Language. To improve the areas where other models have limitation (e.g. paralleliza tion, Long-range dependency, sequential computing, positional encoding, scalability and Network) our research suggests an Attention-based Bidirectional Transformer Encoder model for recognizing objectionable content in local Urdu language to fill this gap. The effectiveness of our suggested approach is compared to the above-mentioned research, taking into account evaluation criteria, dataset size, and the word embedding layer’s influence. Pre-trained Urdu Word2Vec embeddings are used for our tests. The out comes show that our transformer-based bidirectional approach improves to 85 percent age. Our tests demonstrate the efficiency-improving power of the attention layer while also emphasizing the inadequacy of pre-trained Word2Vec embeddings for the detection of unsuitable content in Urdu datasets.