.png){kind=icon}
.png){kind=icon}
Return
Selective interactive networks with knowledge graphs for image classification
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2023.110889.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The combination of networks and knowledge graphs (KGs) improves the accuracy and interpretability of networks in image classification. Previous methods fuse the entire KG with each sample. However, there is limited knowledge related to specific samples for the entire KG. The fusion process of the previous method includes some useless redundant knowledge, resulting in suboptimal fusion performance. In this paper, we propose a selective interactive network based on a knowledge graph, which builds a bidirectional communication bridge for knowledge and data. Specifically, we design an attention based knowledge selection module that utilizes data to select effective knowledge from the KG, filtering out some useless knowledge involved in the subsequent fusion process. Meanwhile, we introduce knowledge selection loss that measures the consistency between knowledge and image features to achieve the effective selection of knowledge through data. Then, we employ a bilinear fusion method to combine the selected knowledge and image features to improve the recognition performance of the network. Extensive experiments conducted on the IDRiD and CUB-200-2011 datasets demonstrate the stateof-the-art performance of the proposed method, especially on the small dataset IDRiD, where the proposed method outperforms the baseline by 6.79% in accuracy. The knowledge selection mechanism establishes the association between predicted classes and the KG, and thus provides attribute descriptions for predicted classes, which has stronger interpretability. The visualization results demonstrate that our model captures more stable image features. & COPY; 2023 Published by Elsevier B.V.
Keywords:
Knowledge graph
Image classification
Integration effect
Knowledge selection
Journal
K
IF:
7.6
Papers:
1.2W
Citations:
4.5W