.png){kind=icon}
返回
Simple But Powerful, a Language-Supervised Method for Image Emotion Classification
.png){kind=icon}
PRE
AI
DOI：10.1109/TAFFC.2022.3225049.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Image emotion classification is an important computer vision task to extract emotions from images. The methods for image emotion classification (IEC) are primarily based on label or distribution as a supervision signal, which neither has enough accessibility nor diversity, limiting the development of IEC research. Inspired by psychology research and the recent booming of large-scale pretrained language models. We figure out a language-supervised paradigm, which can cleverly combine the features of language and visual emotion to drive the visual model to gain stronger emotional discernment with language prompts. To practice the paradigm, we present a conceptually simple while empirically powerful framework for image emotion classification, SimEmotion. That we propose a prompt-based fine-tuning strategy to learn task-specific representations by composing a template with the emotion-level concept and entity-level information. Evaluations on four widely-used affective datasets, namely, Flickr and Instagram (FI), EmotionROI, Twitter I, and Twitter II, demonstrate that the proposed algorithm outperforms the state-of-the-art methods with a large margin (i.e., 8.42% absolute accuracy gain on EmotionROI) on image emotion classification tasks. Our codes will be publicly available for research purposes.
Keyword:
Task analysis
IEC
Wheels
Psychology
Dogs
Training
Visualization
Language-supervised
prompt tuning
image emotion classification
fine-tuning
computer vision
期刊
IF:
9.8
论文数:
1.4K
被引数:
9.1K
机构
引用论文
Simultaneous Dust and Pollutant Transport over East Asia: The Tripartite Environment Ministers Meeting March 2014 Case Study.png)
SOLA
OAAI
SOLAIF0
OAAI分享收藏
分享收藏
查看更多内容