.png){kind=icon}
返回
Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI
.png){kind=icon}
OA
AI
DOI：10.1109/TKDE.2024.3399746.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Embodied AI is one of the most popular studies in artificial intelligence and robotics, which can effectively improve the intelligence of real-world agents (i.e. robots) serving human beings. Scene knowledge is important for an agent to understand the surroundings and make correct decisions in the varied open world. Currently, knowledge base for embodied tasks is missing and most existing work use general knowledge base or pre-trained models to enhance the intelligence of an agent. For conventional knowledge base, it is sparse, insufficient in capacity and cost in data collection. For pre-trained models, they face the uncertainty of knowledge and hard maintenance. To overcome the challenges of scene knowledge, we propose a scene-driven multimodal knowledge graph (Scene-MMKG) construction method combining conventional knowledge engineering and large language models. A unified scene knowledge injection framework is introduced for knowledge representation. To evaluate the advantages of our proposed method, we instantiate Scene-MMKG considering typical indoor robotic functionalities (Manipulation and Mobility), named ManipMob-MMKG. Comparisons in characteristics indicate our instantiated ManipMob-MMKG has broad superiority on data-collection efficiency and knowledge quality. Experimental results on typical embodied tasks show that knowledge-enhanced methods using our instantiated ManipMob-MMKG can improve the performance obviously without re-designing model structures complexly.
Keyword:
Task analysis
Knowledge graphs
Artificial intelligence
Knowledge based systems
Robots
Knowledge engineering
Visualization
Multimodal knowledge graph
scene driven
embodied AI
robotic intelligence
期刊
IF:
10.4
论文数:
6.8K
被引数:
3.2W
机构
引用论文
Kinematic analysis of limb movements in neuropsychological research: Subtle deficits and recovery of function.神经心理学研究中肢体运动的运动学分析: 细微的缺陷和功能的恢复。
PREAI
分享收藏Substrate specificity and inhibitor analyses of human steroid 5β-reductase (AKR1D1)人类类固醇5 β-还原酶 (AKR1D1) 的底物特异性和抑制剂分析.png)
Steroids
OAAI
SteroidsIF0
OAAI分享收藏分享收藏DBpedia - A large-scale, multilingual knowledge base extracted from WikipediaDBpedia-从维基百科中提取的大规模多语言知识库SEMANTIC WEBOAAI
SEMANTIC WEBIF2.9
OAAI分享收藏Communication and reproductive behaviour in North American Jerusalem crickets (
Stenopelmatus
) (Orthoptera: Stenopelmatidae).北美耶路撒冷蟋蟀的交流和生殖行为 (
Stenopelmatus
) (直翅目: Stenopelmatidae)。
PREAI
分享收藏
查看更多内容