SI-Net: spatial interaction network for deepfake detection

Jian Wang

doi:10.1007/s00530-023-01114-w

2023-07-01

PRE

DOI：10.1007/s00530-023-01114-w

原文链接

原文求助

摘要

En 中文

As manipulated faces become more realistic and indistinguishable, there is a high demand for efficiently and accurately detecting deepfakes. Existing CNN-based deepfake detection methods either learn a global feature representation of the whole face or learn multiple local features. However, these methods learn the global and local features independently, thus neglect the spatial correlations between the local features and global context, which are vital in identifying different forgery patterns. Therefore, in this paper, we propose Spatial Interaction Network (SI-Net), a deepfake detection method to mine potential complementary and co-occurrent features between local texture and global context concurrently. Specifically, we first utilize a region feature extractor that distills local features from the global features, to simplify the procedure of local feature extraction. We then propose spatial-aware transformer to learn the co-occurrence feature from local texture and global context, concurrently. We capture the attended feature from the local regions according to their importance. The final prediction is made through the composite considerations of the aforementioned modules. Experimental results on two public datasets, FaceForensics++ and WildDeepfake, demonstrate the superior performance of SI-Net compared with the state-of-the-art methods.

Keyword:

Deepfake detection

Spatial interaction

Self-attention mechanism