.png){kind=icon}
Return
Skill-Aware Diffusion for Generalizable Robotic Manipulation
.png){kind=icon}
PRE
AI
DOI：10.1109/tase.2026.3740160.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Robust generalization in robotic manipulation is crucial for robots to adapt flexibly to diverse environments. Existing methods usually improve generalization by scaling data and networks, but model tasks independently and overlook skill-level information. Observing that tasks within the same skill share similar motion patterns, we propose Skill-Aware Diffusion (SADiff), which explicitly incorporates skill-level information to improve generalization. SADiff learns skill-specific representations through a skill-aware encoding module with learnable skill tokens, and conditions a skill-constrained diffusion model to generate object-centric motion flow. A skill-retrieval transformation strategy further exploits skill-specific trajectory priors to refine the mapping from 2D motion flow to executable 3D actions. Furthermore, we introduce IsaacSkill, a high-fidelity dataset containing fundamental robotic skills for comprehensive evaluation and sim-to-real transfer. Experiments in simulation and real-world settings show that SADiff achieves good performance and generalization across various manipulation tasks. Dataset and videos are available at https://sites.google.com/view/sa-diff Note to Practitioners—This research aims to address the issue that existing methods tend to overlook shared skill-level motion patterns when generalizing robotic manipulation to diverse environments. The proposed approach introduces skill-level modeling to capture shared motion patterns in related tasks, improving adaptability to new scenarios. By combining skill-aware representations with a diffusion-based generation framework, the method produces more consistent motion flow even in unfamiliar scenarios. In addition, a skill-retrieval strategy further enhances the robustness of the mapping between motion flow and action. Our method has been validated through comprehensive experiments in simulated and real-world settings.
Keywords:
Robotic manipulation
motion planning
diffusion model
skill transfer
Journal
IF:
6.4
Papers:
5.1K
Citations:
1.6W
Organization
View more organizations
Cited Papers
No cited papers available