Prior knowledge guided text to image generation

An-An Liu

doi:10.1016/j.patrec.2023.12.003

Prior knowledge guided text to image generation

2024-01-01

7

PRE

AI

DOI：10.1016/j.patrec.2023.12.003

Original

Original request for help

Save

Abstract

En 中文

Generating a realistic and semantically consistent image from a given text is a challenging task. Due to the limited information of natural language, it is difficult to generate vivid images with fine details. To address this problem, we propose a Prior Knowledge Guided GAN for text to image generation. Specifically, the proposed method consists of several Knowledge Guided Up-Blocks. We decompose the image into a superposition of several visual regions, each of which requires corresponding prior knowledge to enrich its visual details. Correspondingly, we construct each Up-Block by incorporating relevant prior knowledge as input, aiming to enhance the quality of each visual region. Prior knowledge progressively provides more visual detail through affine transformations. Finally, high-quality images are synthesized by fusing all image regions. Experimental results on the CUB and COCO datasets demonstrate the superior performance of the proposed method.

Keywords:

Text-to-image synthesis

Generative Adversarial Networks

Knowledge Guided GAN

Prior knowledge guided text to image generation

Abstract

Journal

Organization