.png){kind=icon}
.png){kind=icon}
Return
“Set it up”: Functional object arrangement with compositional generative models
.png){kind=icon}
PRE
AI
DOI：10.1177/02783649251378198.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Functional object arrangement
(FORM) is the task of arranging objects to fulfill a function, for example, “set up a dining table for two.” One key challenge here is that the instructions for FORM are often under-specified and do not explicitly specify the desired object goal poses. This paper presents
SetItUp
, a neuro-symbolic framework that learns to specify the goal poses of objects from a few training examples and a structured natural-language task specification. SetItUp uses a
grounding graph
, which is composed of abstract spatial relations among objects (
e.g
.,
left
-
of
), as its intermediate representation. This decomposes the FORM problem into two stages: (i) predicting this graph among objects and (ii) predicting object poses given the grounding graph. For (i), SetItUp leverages large language models (LLMs) to induce Python programs from a task specification and a few training examples. This program can be executed to generate grounding graphs in novel scenarios. For (ii), SetItUp pre-trains a collection of diffusion models to capture primitive spatial relations and online composes these models to predict object poses based on the grounding graph. We evaluated SetItUp on a dataset spanning three distinct task families: arranging tableware on a dining table, organizing items on a bookshelf, and laying out furniture in a bedroom. Experiments show that SetItUp outperforms existing models in generating functional, physically feasible, and aesthetically pleasing object arrangements.
Journal
T
IF:
0
Papers:
126
Citations:
0