Generalized face forgery detection with self-supervised face geometry information analysis network

Rui Zhang

doi:10.1016/j.asoc.2024.112143

Generalized face forgery detection with self-supervised face geometry information analysis network

2024-11-01

0

PRE

AI

DOI：10.1016/j.asoc.2024.112143

原文链接

原文求助

摘要

En 中文

The emergence of high-quality deepfake facial videos has raised concerns about the security of facial images. Existing face forgery detectors mainly tend to locate a specific forgery region of the human face for detection, which achieves satisfactory performance with known forgery patterns presented in the training set. However, with the continuous advancements in face forgery technology, this approach becomes less reliable with new forgery patterns that emerge. Towards this end, we proposed a novel S elf-supervised F ace G eometry Information A nalysis N etwork (SF-GAN) method for generalized face forgery detection. SF-GAN effectively leverages the relationships among informative regions based on information theory. Drawing from information theory, regions with high uncertainty tend to contain more valuable information. Our methodology integrates a self-supervised learning mechanism, enabling the precise identification of multiple informative regions. Furthermore, we leverage facial geometry by establishing both explicit and latent geometric relationships through the use of Graph Convolutional Networks (GCNs). Within our framework, facial landmarks and informative regions are depicted as nodes in the GCNs. By analyzing the geometric relationships between the graph of facial landmarks and the graph of informative regions, we are able to identify valid anomalous regions, thereby minimizing uncertainty. Our proposed model gains a comprehensive understanding of common information in face forgery images. Extensive experiments on eight large-scale benchmark datasets: FaceForensics++ (FF++), WildDeepfake (WDF), Celeb-DF v2 (CDF), DeepFake Detection Challenge (DFDC), DFDC preview (DFDC-P), Deepfake Detection (DFD), DeeperForensics-1.0 (DF-1.0) and ForgeryNIR, show that the proposed method is comparable to state-of-the-arts and exhibits better generalizability. Specifically, our SF-GAN, when trained on high-quality FF++ data, achieves an impressive AUC of 76.43% on the CDF dataset.

Keyword:

Face forgery detection

Self-supervision mechanism

Graph convolutional networks

Information theory

Generalized face forgery detection with self-supervised face geometry information analysis network

摘要

期刊

机构

引用论文