.png){kind=icon}
返回
Visual question answering by pattern matching and reasoning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2021.10.016.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Traditional techniques for visual question answering (vQA) are mostly end-to-end neural network based, which often perform poorly (e.g., low accuracy) due to lack of understanding and reasoning. To overcome the weaknesses, we propose a comprehensive approach with following key features. (1) It represents inputs, i.e., an image f and a natural language question Q(nl) as an entity-attribute graph and a query graph, respectively, and employs pattern matching to find answers; (2) it leverages reinforcement learning based model to identify a set of policies that are used to guide visual tasks and construct an entity-attribute graph, based on Q(nl); (3) it employs a novel method to parse a question Q(nl) and generate corresponding query graph Q(u(o)) for pattern matching; and (4) it integrates inference scheme to further improve result accuracy, in particular, it learns a graph-structured classifier for missing value inference and a co-occurrence matrix for candidate selection. With these features, our approach can not only process visual tasks efficiently, but also answer questions with high accuracy. To evaluate the performance of our approach, we conduct empirical studies on Soccer, Visual-Genome and GQA, and show that our approach outperforms the state-of-the-art methods in result accuracy and system efficiency. (C) 2021 Elsevier B.V. All rights reserved.
Keyword:
Visual question answering
Reinforcement learning
Inference
Pattern matching
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
分享收藏Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
OAAI
OAAI分享收藏分享收藏Accurate parameter estimation for Bayesian network classifiers using hierarchical Dirichlet processes.png)
MACHINE LEARNINGOAAI
MACHINE LEARNINGIF2.9
OAAI分享收藏
查看更多内容