.png){kind=icon}
返回
Graph Pooling Inference Network for Text-based VQA
.png){kind=icon}
PRE
AI
DOI：10.1145/3634918.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Effectively leveraging objects and optical character recognition (OCR) tokens to reason out pivotal scene text is critical for the challenging Text-based Visual Question Answering (TextVQA) task. Graph-based models can effectively capture the semantic relationship among visual entities (objects and tokens) and report remarkable performance in TextVQA. However, previous efforts usually leverage all visual entities and ignore the negative effect of superfluous entities. This article presents a Graph Pooling Inference Network (GPIN), which is an evolutionary graph learning method to purify the visual entities and capture the core semantics. It is observed that the dense distribution of reduplicative objects and the crowd of semantically dependent OCR tokens usually co-exist in the image. Motivated by this, GPIN adopts an adaptive node dropping strategy to dynamically downscale semantically closed nodes for graph evolution and update. To deepen the comprehension of scene text, GPIN is a dual-path hierarchical graph architecture that progressively aggregates the evolved object graph and the evolved token graph semantics into a graph vector that serves as visual cues to facilitate the answer reasoning. It can effectively eliminate object redundancy and enhance the association of semantically continuous tokens. Experiments conducted on TextVQA and ST-VQA datasets show that GPIN achieves promising performance compared with state-of-the-art methods.
Keyword:
Text-based visual question answering
graph inference
graph pooling
期刊
IF:
6
论文数:
2.0K
被引数:
5.4K
机构
查看更多机构
引用论文
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
OAAI
.png){kind=icon}
OAAI分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏分享收藏X-ray investigation of the coordination and complex formation of lanthanoid ions in aqueous perchlorate and selenate solutions高氯酸盐和硒酸水溶液中镧系离子的配位和络合物形成的x射线研究
PREAI
分享收藏Inosine pranobex enhances human NK cell cytotoxicity by inducing metabolic activation and NKG2D ligand expression肌苷pranobex通过诱导代谢活化和NKG2D配体表达增强人NK细胞的细胞毒性OAAI
OAAI分享收藏
没有更多内容