TI2Net: Temporal Identity Inconsistency Network for Deepfake Detection

In this paper, we propose the Temporal Identity Inconsistency Network (TI(2)Net), a Deepfake detector that focuses on temporal identity inconsistency. Specifically, TI(2)Net recognizes fake videos by capturing the dissimilarities of human faces among video frames of the same identity. Therefore, TI(2)Net is a reference-agnostic detector and can be used on unseen datasets. For a video clip of a given identity, identity information in all frames will first be encoded to identity vectors. TI(2)Net learns the temporal identity embedding from the temporal difference of the identity vectors. The temporal embedding, representing the identity inconsistency in the video clip, is finally used to determine the authenticity of the video clip. During training, TI(2)Net incorporates triplet loss to learn more discriminative temporal embeddings. We conduct comprehensive experiments to evaluate the performance of the proposed TI(2)Net. Experimental results indicate that TI(2)Net generalizes well to unseen manipulations and datasets with unseen identities. Besides, TI(2)Net also shows robust performance against compression and additive noise.

Automated summary

In this paper, the Temporal Identity Inconsistency Network (TI(2)Net) is proposed as a Deepfake detector that focuses on temporal identity inconsistency. It recognizes fake videos by capturing the dissimilarities of human faces among video frames of the same identity. TI(2)Net is a reference-agnostic detector and can be used on unseen datasets.