.png){kind=icon}
返回
AVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake
.png){kind=icon}
PRE
AI
DOI：10.1109/TIFS.2023.3262148.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, deepfakes have raised severe concerns about the authenticity of online media. Prior works for deepfake detection have made many efforts to capture the intra-modal artifacts. However, deepfake videos in real-world scenarios often consist of a combination of audio and visual. In this paper, we propose an Audio-Visual Joint Learning for Detecting Deepfake (AVoiD-DF), which exploits audio-visual inconsistency for multi-modal forgery detection. Specifically, AVoiD-DF begins by embedding temporal-spatial information in Temporal-Spatial Encoder. A Multi-Modal Joint-Decoder is then designed to fuse multi-modal features and jointly learn inherent relationships. Afterward, a Cross-Modal Classifier is devised to detect manipulation with inter-modal and intra-modal disharmony. Since existing datasets for deepfake detection mainly focus on one modality and only cover a few forgery methods, we build a novel benchmark DefakeAVMiT for multi-modal deepfake detection. DefakeAVMiT contains sufficient visuals with corresponding audios, where any one of the modalities may be maliciously modified by multiple deepfake methods. The experimental results on DefakeAVMiT, FakeAVCeleb, and DFDC demonstrate that the AVoiD-DF outperforms many state-of-the-arts in deepfake detection. Our proposed method also yields superior generalization on various forgery techniques.
Keyword:
Deepfakes
Visualization
Forgery
Detectors
Feature extraction
Faces
Electronic mail
Deepfake detection
multi-modal
audio-visual
joint learning
期刊
IF:
8
论文数:
5.3K
被引数:
2.3W
机构
引用论文
分享收藏
AVFakeNet: A unified end-to-end Dense Swin Transformer deep learning model for audio-visual deepfakes detectionAVFakeNet: 用于视听deepfakes检测的统一端到端密集Swin Transformer深度学习模型
PREAI
分享收藏Genome-Wide Analysis and Exploration of WRKY Transcription Factor Family Involved in the Regulation of Shoot Branching in Petunia.png)
Genes
OAAI
GenesIF0
OAAI分享收藏
查看更多内容