Visually evoked brain signals guided image regeneration using GAN variants

Generative Adversarial Networks have recently proven to be very effective in generative applications involving images, and they are now being used to regenerate images using visually evoked brain signals. Recent neuroscience research has discovered evidence that brain-evoked data can be used to decipher how the human brain functions. Simultaneously, the latest advancement in deep learning integrated with a high-level interest in generative methods has made learning the data distribution possible and realistic images can be produced from random noise. In this work, an advanced generative adversarial method that incorporates the capsule network with the generative adversarial networks model i.e. Capsule Generative Adversarial Network is proposed to regenerate images with decoded information and formulated features from visually evoked brain signals. There are two stages in the proposed method: Encoder, for data formulation of visually evoked brain activity and the image reconstruction phase from the brain signals. The image regeneration technique has been experimentally tested on a variety of generative adversarial networks including the proposed model and the final reconstructed image samples are compared to assess the quality using various evaluation metrics. The Structural Similarity Index Measure metric for Capsule Generative Adversarial Network has achieved highest value i.e., 0.9203 and outperforms the other GAN variants and also indicates that the Capsule Generative Adversarial Network reconstructed the images similar to original images.

Automated summary

Generative Adversarial Networks (GANs) have been shown to be effective in image generation and are now being applied to regenerate images using brain signals. Recent neuroscience research has discovered that brain-evoked data can reveal how the human brain functions. This study proposes a advanced approach, called Capsule Generative Adversarial Network, that combines the capsule network with the GAN model to regenerate images using decoded brain signals. Experimental results demonstrate that the Capsule GAN achieved the highest Structural Similarity Index Measure (0.9203) among various GAN variants, indicating its ability to reconstruct images similar to the originals.