.png){kind=icon}
.png){kind=icon}
Return
Language-Embedded 6D Pose Estimation for Tool Manipulation
.png){kind=icon}
PRE
AI
DOI：10.1109/LRA.2025.3587559.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Robotic tool manipulation requires understanding task-relevant semantics under visually challenging conditions, such as shape variation and occlusion. This paper presents a novel framework for Language-Embedded Semantic 6D Pose Estimation that combines natural language instructions with 3D point cloud data to achieve category-level 6D pose estimation of tools' functional parts. By embedding semantic information from large language models (LLMs) and leveraging a diffusion-based pose estimator, our approach achieves robust generalization across diverse tool categories. We introduce a comprehensive synthetic dataset, tailored for tool manipulation scenarios, with annotated 6D poses of functional parts. Extensive experiments conducted on both the synthetic dataset and real-world robots demonstrate our system's ability to interpret natural language commands, predict poses of functional parts, and perform manipulation tasks with significant improvements in accuracy and generalization.
Keywords:
6D Pose estimation
dataset generation
large language model (LLM)
robotic manipulation
Journal
I
IF:
5.3
Papers:
1.7K
Citations:
3.9W