.png){kind=icon}
返回
Decomposed Prototype Learning for Few-Shot Scene Graph Generation
.png){kind=icon}
OA
AI
DOI：10.1145/3700877.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Today's scene graph generation (SGG) models typically require abundant manual annotations to learn new predicate types. Therefore, it is difficult to apply them to real-world applications with massive uncommon predicate categories whose annotations are hard to collect. In this article, we focus on Few-Shot SGG (FSSGG), which encourages SGG models to be able to quickly transfer previous knowledge and recognize unseen predicates well with only a few examples. However, current methods for FSSGG are hindered by the high intra-class variance of predicate categories in SGG: On one hand, each predicate category commonly has multiple semantic meanings under different contexts. On the other hand, the visual appearance of relation triplets with the same predicate differs greatly under different subject-object compositions. Such great variance of inputs makes it hard to learn generalizable representation for each predicate category with current few-shot learning (FSL) methods. However, we found that this intra-class variance of predicates is highly related to the composed subjects and objects. To model the intra-class variance of predicates with subject-object context, we propose a novel Decomposed Prototype Learning (DPL) model for FSSGG. Specifically, we first construct a decomposable prototype space to capture diverse semantics and visual patterns of subjects and objects for predicates by decomposing them into multiple prototypes. Afterwards, we integrate these prototypes with different weights to generate query-adaptive predicate representation with more reliable semantics for each query sample. We conduct extensive experiments and compare with various baseline methods to show the effectiveness of our method.
Keyword:
Scene Graph Generation (SGG)
Few-Shot Learning
Prompt Learning
Prototype Learning
期刊
IF:
6
论文数:
2.0K
被引数:
5.4K
机构
引用论文
Transcription initiation of distant core promoters in a large-sized genome of an insect.png)
BMC Biology
OAAI
BMC BiologyIF0
OAAI分享收藏Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉OAAI
OAAI分享收藏分享收藏The adsorption of monolayer coatings on iron nanoparticles: Mössbauer spectroscopy and XANES results
PREAI
分享收藏
查看更多内容