Abstract:
Reconstructing fine features of natural images from functional Magnetic Resonance
Imaging (fMRI) of never-before-seen data is challenging. Acquiring a large-scale fMRI
from a subject viewing natural images is a highly resource-intensive endeavor. Due to
limited sample data, the recent approaches use a pre-trained generative network or unsupervised learning method for extracting features of reconstructing natural images during
network training. However, these reconstructed images are not clear and unidentifiable
unless their ground truths are available. Further, these results cannot be enhanced up
to identifiable natural images because the training model will hardly learn to construct
coarse features of never-before-seen data. However, it was observed from the state-ofthe-art results that the reconstructed natural images exhibit consistent structure, shape,
and coarse features across subjects although each subject’s fMRI is different. Leveraging nearly uniform reconstructed test images across subjects, we reorient our research
focus. The proposed approach utilizes Beliy’s model to reconstruct the images from the
test subject’s fMRI followed by the proposed denoising model, based on the pre-trained
classification model and generative adversarial Network BigGAN, to produce identifiable images. Our proposed approach using our own denoising model gives up to 70%
identification accuracy in comparison with the 16%, and 30% identification accuracy
given by the adversarial autoencoder and variational autoencoder, respectively.
