Efficient and Controllable Remote Sensing Fake Sample Generation Based on Diffusion Model

doi:10.1109/TGRS.2023.3268331

2023-01-01

PRE

DOI：10.1109/TGRS.2023.3268331

原文链接

原文求助

摘要

En 中文

In this article, we propose an efficient remote sensing fake sample generation (RSFSG) framework based on the diffusion model, so as to generate controllable samples consistent with real scenes. Firstly, in order to alleviate the huge time consumption caused by the large parameters of the diffusion model, we come up with a multifrequency dynamic knowledge distillation based on the consistent power spectrum of predicted Gaussian noise. The proposed multi-frequency knowledge transfer lets the lightweight model learn different frequency outputs from teacher model during diffusion process at different stages. Secondly, to address the problem of slow training of diffusion models, we propose a progressive training strategy (PTS), inspired by the fitting mechanism of deep networks from low to high frequencies. PTS enables fast fitting of diffusion models by enabling the model to learn from low-frequency information such as color at low resolution, and gradually move to high-resolution images full of details such as texture. The established two methods above achieve good generation performance under lightweight parameters, within almost half of the training time consumed. Extensive evaluations demonstrate that the proposed model significantly outperforms the state-of-the-art methods on RS controllable fake sample generation. Within our knowledge, we are the first to introduce the diffusion model into the RSFSG task and obtain good performance; the code and the corresponding pretrained files have been released at https://github.com/xiaoyuan1996/Controllable-Fake-Sample-Generation-for-RS.

Keyword:

Diffusion model

multifrequency dynamic diffusion knowledge distillation

progressive training strategy (PTS) for accelerated diffusion learning

remote sensing fake sample generation (RSFSG)