Multimodal 3D-MRI Brain Tumor Segmentation via Modified 3D U-Net with Transformer

Abstract

Accurate brain tumor segmentation in 3D magnetic resonance imaging (MRI) scans is pivotal for medical diagnosis and treatment planning. This master’s thesis introduces an advanced approach for multimodal 3D-MRI brain tumor segmentation. Our methodology combines a modified 3D U-Net architecture with Transformer-based self-attention, and dilated convolution layers. By leveraging multimodal information from T1-weighted, T2-weighted, T1 Contrast Enhanced (T1CE) Imaging and FLAIR MRI sequences within the BraTS 2023 dataset, our method significantly enhances segmentation precision. The modified 3D U-Net with Transformer and dilated convolution layers enables effective capture of both local and global contextual information, facilitating the identification of complex structures and long-range dependencies within volumetric MRI data. Thorough experimentation and evaluation, including comparisons with 3D U-Net and its variants, highlight the superiority of our proposed model in terms of brain tumor delineation accuracy. These findings emphasize the potential of this hybrid architecture to advance medical image analysis, providing substantial benefits to patient care.