.png){kind=icon}
返回
Learning to Reason on Tree Structures for Knowledge-Based Visual Question Answering
.png){kind=icon}
OA
AI
DOI：10.3390/s22041575.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Collaborative reasoning for knowledge-based visual question answering is challenging but vital and efficient in understanding the features of the images and questions. While previous methods jointly fuse all kinds of features by attention mechanism or use handcrafted rules to generate a layout for performing compositional reasoning, which lacks the process of visual reasoning and introduces a large number of parameters for predicting the correct answer. For conducting visual reasoning on all kinds of image-question pairs, in this paper, we propose a novel reasoning model of a question-guided tree structure with a knowledge base (QGTSKB) for addressing these problems. In addition, our model consists of four neural module networks: the attention model that locates attended regions based on the image features and question embeddings by attention mechanism, the gated reasoning model that forgets and updates the fused features, the fusion reasoning model that mines high-level semantics of the attended visual features and knowledge base and knowledge-based fact model that makes up for the lack of visual and textual information with external knowledge. Therefore, our model performs visual analysis and reasoning based on tree structures, knowledge base and four neural module networks. Experimental results show that our model achieves superior performance over existing methods on the VQA v2.0 and CLVER dataset, and visual reasoning experiments prove the interpretability of the model.
Keyword:
tree structure
knowledge base
compositional reasoning
neural module network
attention mechanism
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.5
论文数:
7.2W
被引数:
20.9W
机构
引用论文
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering使VQA中的V重要: 提升图像理解在视觉问答中的作用OAAI
OAAI分享收藏分享收藏分享收藏
分享收藏Cobalt and Copper Composite Oxides as Efficient Catalysts for Preferential Oxidation of CO in H2-Rich Stream钴和铜复合氧化物作为H2-Rich流中CO优先氧化的有效催化剂
PREAI
分享收藏A Mild Protocol for the Regioselective Ring Opening of Doubly Activated Cyclopropanes by Using Selenolates Generated in Situ: Synthesis of Functionalized Organoselenium Compounds.png)
Synthesis
SynthesisIF0
PREAI
分享收藏
查看更多内容