.png){kind=icon}
Return
A visual question answering model based on entity knowledge selection
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-026-02627-w.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
One of the key limitations in knowledge-based Visual Question Answering (VQA) is the inability of existing models to extract precise knowledge from structured bases for complex queries. To address this issue, we propose Entity Knowledge Selection (EKS), a novel framework that leverages entity relations in commonsense knowledge graphs to dynamically generate knowledge sentences relevant to both visual and textual entities. EKS formulates knowledge selection as a relevance scoring problem, where semantic similarity is used to measure the relevance between knowledge candidates and the input question. In addition, we introduce a Multimodal Feature Encoding (MFE) strategy to extract image region features aligned with the question. The selected knowledge is integrated with visual and linguistic features through the Knowledge Multi-feature Fusion module (KMF) within a unified VL-BERT framework for cross-modal interaction. Extensive experiments on the OK-VQA dataset demonstrate that our method outperforms several representative knowledge-based VQA approaches and achieves competitive performance among methods that do not rely on extremely large language models.
Keywords:
Visual question answering
Entity knowledge selection
Entity relationship
VL-BERT
Journal
IF:
3.1
Papers:
2.8K
Citations:
2.7K
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSaveShareSaveShareSave
View more