.png){kind=icon}
返回
DeformSg2im: Scene graph based multi-instance image generation with a deformable geometric layout
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2023.126684.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, scene graph based image generation has emerged to be an important research direction for advanced multi-instance generation tasks. Scene layout generation, the phase that produces the instance-wise visual representations and maintains the spatial relationships among all instances, is vital in translating a scene graph to an image. However, the scene layouts generated by existing methods are too coarse and semantically inconsistent with the given scene graph, which results in quality degradation of the generated images. Motivated by this, we propose a novel scene graph based image generation model called DeformSg2im, which aims to generate visually appealing and semantically faithful images according to the given scene graphs. Our method addresses the aforementioned problems from two aspects. In one aspect, we present an attention-based instance embedding estimator to refine the shape information for each instance. By introducing attention maps to the estimation of instance embeddings, our method is able to generate more plausible instances with sharp edges on the images. In the other aspect, a spatial warping network (SWN) is proposed to adaptively capture the spatial dependencies among instances. With sequential modeling and geometric deformations, the SWN is capable of generating the scene layout conforming to the given scene graph. Extensive experiments show that our model generates images with high visual quality and achieves competitive quantitative results compared to existing works. Ablation studies on the proposed modules are also conducted, and the results demonstrate the effectiveness of our method.
Keyword:
Scene graph
Multi-instance image generation
Attention-based mask synthesis
Sequential modeling
Geometric deformation
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
.png){kind=icon}
Cushing syndrome induced by long term use of corticosteroids in the management of atopic dermatitis由长期使用皮质类固醇治疗特应性皮炎引起的库欣综合征
OAAI
OAAI分享收藏Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
OAAI
OAAI分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏
查看更多内容