Abstract:
The automated generation of radiology reports provided X-rays and has tremendous potential
to operationally enhance the clinical diagnosis of diseases for patients. A new landmark of
research is being created by using hybrid approaches based on Natural Language Processing
and Computer Vision Techniques to create auto medical report generation systems. The auto
report generator, producing radiology reports will significantly reduce the burden for doctors
and assist them in writing manual reports. Because of the sensitivity of the findings of the Chest
X-rays (CXR) provided by the existing techniques are not adequately accurate. Therefore,
producing comprehensive explanations for medical photographs remains a difficult task. A
novel approach to address this issue is proposed. It is based on continuous integration of
Convolution Neural Networks (CNNs) for detecting diseases and Long Short-Term Memory
(LSTM) followed by Attention Mechanism for sequence generation based on those diseases.
Experimental results obtained by utilizing Indiana University (IU) CXR dataset showed that
the suggested model attains the current state-of-the-art efficiency as opposed to other solutions
to the baseline. Some experiments were also performed without using attention block. As
evaluation metric, BELU-0, BELU-1, BELU-2 and BELU-3 have been applied. The score we
got without the attention Mechanism are 0.522, 0.262, 0.201, 0.119. With attention Mechanism
the state of the art BELU scores are 0.580, 0.342, 0.263 and 0.155. Red heat maps also attained
that not only captured the diseases present in an image, but it also expresses how these diseases
are related to each other as well as their attributes and the activities they are involved in.
