.png){kind=icon}
返回
Action-conditional implicit visual dynamics for deformable object manipulation
.png){kind=icon}
PRE
AI
DOI：10.1177/02783649231191222.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, brings substantial challenges due to infinite shape variations, non-rigid motions, and partial observability. We introduce ACID, an action-conditional visual dynamics model for volumetric deformable objects based on structured implicit neural representations. ACID integrates two new techniques: implicit representations for action-conditional dynamics and geodesics-based contrastive learning. To represent deformable dynamics from partial RGB-D observations, we learn implicit representations of occupancy and flow-based forward dynamics. To accurately identify state change under large non-rigid deformations, we learn a correspondence embedding field through a novel geodesics-based contrastive loss. To evaluate our approach, we develop a simulation framework for manipulating complex deformable shapes in realistic scenes and a benchmark containing over 17,000 action trajectories with six types of plush toys and 78 variants. Our model achieves the best performance in geometry, correspondence, and dynamics predictions over existing approaches. The ACID dynamics models are successfully employed for goal-conditioned deformable manipulation tasks, resulting in a 30% increase in task success rate over the strongest baseline. Furthermore, we apply the simulation-trained ACID model directly to real-world objects and show success in manipulating them into target configurations.
Keyword:
Deformable manipulation
implicit neural representations
visual dynamics
期刊
IF:
5
论文数:
2.4K
被引数:
1.6W
机构
引用论文
.png){kind=icon}
On the visual deformation servoing of compliant objects: Uncalibrated control methods and experiments关于顺应对象的视觉变形伺服: 未校准的控制方法和实验
PREAI
分享收藏分享收藏分享收藏
查看更多内容