4.7 Article

Model-Agnostic Method: Exposing Deepfake Using Pixel-Wise Spatial and Temporal Fingerprints

期刊

IEEE TRANSACTIONS ON BIG DATA
卷 9, 期 6, 页码 1496-1509

出版社

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TBDATA.2023.3284272

关键词

Auto-regressive (AR); deep learning; deepfake detection; fingerprint; photoplethysmography (PPG); temporal and spatial

向作者/读者索取更多资源

This paper presents a method for exposing Deepfake by analyzing temporal and spatial faint synthetic signals hidden in portrait videos. By extracting PPG features and AR features, the method can effectively detect fake content. Experimental results show that this method achieves state-of-the-art performance on multiple datasets and demonstrates better generalization.
Deepfake poses a serious threat to the reliability of judicial evidence and intellectual property protection. Existing detection methods either blindly utilize deep learning or use biosignal features, but neither considers spatial and temporal relevance of face features. These methods are increasingly unable to resist the growing realism of fake videos and lack generalization. In this paper, we identify a reliable fingerprint through the consistency of AR coefficients and extend the original PPG signal to 3-dimensional fingerprints to effectively detect fake content. Using these reliable fingerprints, we propose a novel model-agnostic method to expose Deepfake by analyzing temporal and spatial faint synthetic signals hidden in portrait videos. Specifically, our method extracts two types of faint information, i.e., PPG features and AR features, which are used as the basis for forensics in temporal and spatial domains, respectively. PPG allows remote estimation of the heart rate in face videos, and irregular heart rate fluctuations expose traces of tampering. AR coefficients reflect pixel-wise correlation and spatial traces of smoothing caused by up-sampling in the process of generating fake faces. Furthermore, we employ two ACBlock-based DenseNets as classifiers. Our method provides state-of-the-art performance on multiple deep forgery datasets and demonstrates better generalization.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据