.png){kind=icon}
.png){kind=icon}
Return
Deception Detection Meets Vision-Language Models
.png){kind=icon}
PRE
AI
DOI：10.1007/s11263-026-02932-x.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Pretrained vision-language models like CLIP have recently demonstrated outstanding performance across various downstream tasks, including image classification and video understanding. However, in high-level visual tasks like automated deception detection (ADD), the semantic association between labels and visual content is weak and lacks explicit textual descriptions. Consequently, the applicability of such models to automated deception detection remains uncertain. To address this question, this paper applies the pre-trained CLIP model to automated deception detection and proposes a new ADD-CLIP model. Inspired by findings in psychology, we inject domain knowledge of deception behavior into the CLIP model. Specifically, ADD-CLIP consists of a visual part and a textual part. In the visual part, we introduce two prompt learning strategies in the image encoder of CLIP: temporal and micro-difference prompt learning. These strategies are implemented through learnable network layers to capture the temporal dynamics and subtle motion characteristics of faces. In the text part, instead of using traditional category labels as text modality input, we build facial behavior difference textual descriptions from both face and AU perspectives as text supervision with the help of large language model such as ChatGPT. In addition, we add learnable dual prompts corresponding to the behavior descriptions at the facial-level and AU-level, respectively. This design allows the model to automatically learn complementary semantic spaces, resulting in a more accurate decision boundary. Extensive experimental results show that ADD-CLIP achieves strong performance on the DOLOS, Bag-of-Lies, and MU3D datasets, verifying its effectiveness. The code is publicly available at https://github.com/zdl12345678/Deception-Detection-Meets-Vision-Language-Models .
Keywords:
Automated Deception Detection
Deep Learning
Visual Language Model
Journal
IF:
9.3
Papers:
3.9K
Citations:
2.8W