Localization of Inpainting Forgery With Feature Enhancement Network

Inpainting the given region of an image is a typical requirement in computer vision. Conventional inpainting, through exemplar-based or diffusion-based strategies, can create realistic inpainted images at a very low cost. Also, such easy-to-use manipulation poses new security threats. Therefore, the detection of inpainting has attracted considerable attention from researchers. However, the existing methods are typically not suitable for the general detection of various inpainting algorithms. Motivated by this, in this work, an efficient feature enhancement network is proposed to locate the inpainted regions in the digital image. First, we design an artifact enhancement block to effectively capture the traces left by diffusion or exemplar-based inpainting. Then, the VGGNet is used as a feature extractor to describe advanced and low-resolution features. Finally, to take full advantage of enhanced features, we concatenate the features obtained by the feature extractor and the up-sampling operations. Extensive experimental evaluations, covering benchmarking, ablation, robustness, generalization, and efficiency studies, confirm the usefulness of the proposed method. This is especially true on the conventional inpainting dataset, our method obtains an average F1 score 7.63% higher than the second-best method. Theoretical and numerical analyses support the effectiveness of our feature enhancement network in representing the artifacts in inpainted images, exhibiting better potential for real-world forensics than various state-of-the-art strategies.

Automated summary

In this work, an efficient feature enhancement network is proposed to locate the inpainted regions in the digital image. First, an artifact enhancement block is designed to effectively capture the traces left by diffusion or exemplar-based inpainting. Then, the VGGNet is used as a feature extractor to describe advanced and low-resolution features. Extensive experimental evaluations confirm the usefulness of the proposed method.