Transformer based Interpretable Model for Semantic Segmentation of Skin Histology Images

Abstract

This study aims to enhance the generalizability and robustness of the deep learning transformer algorithm employed in segmenting skin cancers and tissues using incremental learning. While deep learning models frequently perform well on tasks identical to their training data, their accuracy suffers substantially when applied to new or different scenarios, exposing limits in generalization. Furthermore, the need for huge annotated datasets is a substantial problem in medical applications, despite the use of techniques such as data augmentation, and transfer learning. To fix such challenges, we propose an incremental learning strategy that improves a model as new data becomes available. This strategy, influenced by the human learning process, seeks to reduce catastrophic forgetting, a typical problem in incremental learning. Our methodology combines loss functions to help integrate new data while keeping existing knowledge. In our studies, the model gets an average accuracy of 89.05% using 10x incremental learning, 92.68% using 10x and 5x learning, and 95.53% with a combination of 10x, 2x, and 5x learning procedures. These results show how our incremental learning methodology improves model performance, reliability, and adaptability for skin cancer segments.