Abstract:
Recent advancements in transformers exploited computer vision problems which
results in state-of-the-art models. Transformer-based models in various sequence
prediction tasks such as language translation, sentiment classification, and caption
generation have shown remarkable performance. Auto report generation scenarios in
medical imaging through caption generation models is one of the applied scenarios for
language models and have strong social impact. In these models, convolution neural
networks have been used as encoder to gain spatial information and recurrent neural
networks are used as decoder to generate caption or medical report. However, using
transformer architecture as encoder and decoder in caption or report writing task
is still unexplored. In this research, we explored the effect of losing spatial biasness
information in encoder by using pre-trained vanilla image transformer architecture
and combine it with different pre-trained language transformers as decoder. In
order to evaluate the proposed methodology, the Indiana University Chest X-Rays
dataset is used where ablation study is also conducted with respect to different
evaluations. The comparative analysis shows that the proposed methodology has
represented remarkable performance when compared with existing techniques in
terms of different performance parameters.
