.png){kind=icon}
.png){kind=icon}
Return
Force-guided multimodal property estimation for robotic scooping of deformable objects
.png){kind=icon}
OA
AI
W
Z
Q
F
J
DOI：10.1016/j.neucom.2026.134734.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Robotic feeding and serving require accurate reasoning about deformable foods, including recognizing food types and estimating the scooped weight. This is challenging because visually similar foods (e.g., millet vs. rice) provide ambiguous appearance cues, while force responses during scooping vary nonlinearly with portion size and utensil dynamics. We propose the Force-guided Text-prompt Multimodal Transformer (FTMT), a framework that integrates vision, force, and language supervision for robust food property estimation. Text prompts are used during training to semantically regularize force features, guiding multimodal tokenization and bi-directional cross-attention between visual and force representations. Experiments on eight deformable food types demonstrate that FTMT achieves improved recognition accuracy and lower weight estimation error compared to force-based and multimodal fusion baselines. Ablation studies highlight the importance of text–force alignment, force preprocessing, and cross-attention fusion. By enabling robots to reliably identify food and predict scooped weight, FTMT represents a step toward adaptive and preference-aware robot-assisted feeding in real-world settings.
Keywords:
Multimodal learning
Multimodal property estimation
Scooping deformable object
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
Cited Papers