.png){kind=icon}
.png){kind=icon}
Return
A Novel Human-Robot Interaction Segmentation: Gesture-Guided Object Segmentation Based on Deep Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/LRA.2026.3656772.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Interactive segmentation, a key branch of human-robot interaction (HRI), bridges human commands to object segmentation, enabling effective robot-scene interaction. However, existing interactive segmentation solutions have significant applicability limitations, particularly when dealing with unknown objects. To this end, this letter proposes a novel deep learning-based interactive segmentation framework by simulating the interaction between human intention and environmental elements, the Pointing Guided Segmentation Network (PGSN), to overcome traditional geometric constraints and establish a direct mapping between pointing gestures and targets. The presented PGSN features a collaborative two-module architecture, with the main contributions: 1) the establishment of direct gesture-to-target mapping and the replacement of conventional click-based interaction methods with pointing gestures; 2) a gesture-pointing-based region prediction module for generating target heatmap; and 3) reconstructed YouRefIt and the self-built PointLab dataset for evaluating gesture-guided segmentation. The effectiveness and feasibility of our proposed method are extensively evaluated on both YouRefIt* and self-built PointLab datasets, achieving an 87.7% accuracy.
Keywords:
Human-robot interaction
interactive object segmentation
deep learning
pointing gesture
prompt
Journal
I
IF:
5.3
Papers:
1.7K
Citations:
3.9W