Wavelet-packets for deepfake image analysis and detection

As neural networks become able to generate realistic artificial images, they have the potential to improve movies, music, video games and make the internet an even more creative and inspiring place. Yet, the latest technology potentially enables new digital ways to lie. In response, the need for a diverse and reliable method toolbox arises to identify artificial images and other content. Previous work primarily relies on pixel-space convolutional neural networks or the Fourier transform. To the best of our knowledge, synthesized fake image analysis and detection methods based on a multi-scale wavelet-packet representation, localized in both space and frequency, have been absent thus far. The wavelet transform conserves spatial information to a degree, allowing us to present a new analysis. Comparing the wavelet coefficients of real and fake images allows interpretation. Significant differences are identified. Additionally, this paper proposes to learn a model for the detection of synthetic images based on the wavelet-packet representation of natural and generated images. Our forensic classifiers exhibit competitive or improved performance at small network sizes, as we demonstrate on the Flickr Faces High Quality, Large-scale Celeb Faces Attributes and Large-scale Scene UNderstanding source identification problems. Furthermore, we study the binary Face Forensics++ (ff++) fake-detection problem.

Automated summary

As neural networks are able to generate realistic artificial images, it becomes crucial to develop methods to identify and analyze these images. This study proposes a novel approach based on wavelet-packet representation for synthesized fake image analysis and detection. The results show that the proposed method achieves competitive performance on small network sizes.