Conditional Generative ConvNets for Exemplar-Based Texture Synthesis

Zi-Ming Wang

doi:10.1109/TIP.2021.3052075

Return

Conditional Generative ConvNets for Exemplar-Based Texture Synthesis

2021-01-01

Zi-Ming Wang

Menghan Li

Gui-Song Xia *

DOI：10.1109/TIP.2021.3052075

Original

Save

View PDF

Abstract

En 中文

The goal of exemplar-based texture synthesis is to generate texture images that are visually similar to a given exemplar. Recently, promising results have been reported by methods relying on convolutional neural networks (ConvNets) pretrained on large-scale image datasets. However, these methods have difficulties in synthesizing image textures with non-local structures and extending to dynamic or sound textures. In this article, we present a conditional generative ConvNet (cgCNN) model which combines deep statistics and the probabilistic framework of generative ConvNet (gCNN) model. Given a texture exemplar, cgCNN defines a conditional distribution using deep statistics of a ConvNet, and synthesizes new textures by sampling from the conditional distribution. In contrast to previous deep texture models, the proposed cgCNN does not rely on pre-trained ConvNets but learns the weights of ConvNets for each input exemplar instead. As a result, cgCNN can synthesize high quality dynamic, sound and image textures in a unified manner. We also explore the theoretical connections between our model and other texture models. Further investigations show that the cgCNN model can be easily generalized to texture expansion and inpainting. Extensive experiments demonstrate that our model can achieve better or at least comparable results than the state-of-the-art methods.

Keywords:

Adaptation models

Image texture

Heuristic algorithms

Probabilistic logic

Maximum likelihood estimation

Generators

Feature extraction

Texture synthesis

deep neural networks

generative model

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.