Attention-aware Feature Fusion based Nuclei Instance Segmentation and Type Classification Using Histology Images

Abstract

The distribution and appearance of nuclei are essential bio markers for the diagnosis and study of cancer. Despite the importance of nuclear morphology accurate segmentation and classification of nuclei instances is still one of the most challenging tasks due to the wide occurrence of overlapping, cluttered nuclei having blurred boundaries. Existing methods particularly focus on region proposal techniques and feature encoding frameworks, however often fail to precisely identify instances. In this paper we propose a simple yet effective model that precisely recognizes instance boundaries as well as caters to exhaustive class imbalance problems, thus yielding accurate class information for each nucleus. We have utilized nuclei pixel positional information i.e its distance from contours for accurate shape estimation along with an object probability score for filtering true nuclei pixels from the background. We have also proposed a novel loss function that draws the same nuclei instance pixels function pulls together for learning an object-based clustering bandwidth thus reinforcing the jaccardian index of the nuclei instance. The network comprises a lightweight attention-aware feature fusion-based architecture having separate instance probability, shape radial estimator, and classification heads. A compound classification loss function is used that minimizes loss by assigning weighted loss to each class according to type occurrence frequency thus mitigating major class imbalance issues existing in most publicly available nuclei datasets.