.png){kind=icon}
.png){kind=icon}
Return
Vector-Quantized Variational AutoEncoder for pansharpening
.png){kind=icon}
PRE
AI
DOI：10.1080/01431161.2023.2265542.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Pansharpening refers to the fusion of a multispectral image (MS) and a panchromatic image (PAN) to obtain a new image with the same spatial resolution as the PAN image and the same spectral resolution as the MS image. This paper describes a new, efficient, and accurate pansharpening architecture. The Vector-Quantized Variational AutoEncoder (VQ-VAE) is the foundation of the proposed method. The VQ-VAE model is trained to learn the non-linear mapping of degraded panchromatic image patches to high-resolution patches. This approach ensures that high-resolution patches can be recovered from low-resolution ones. After training on PAN patches, the VQ-VAE estimates high-resolution multispectral patches for each band of the original multispectral image before reconstructing the high-resolution multispectral image from the patches. The original multispectral image, the panchromatic image, and the estimated high-resolution multispectral image are combined through a modified Component Substitution (CS) process to obtain the pansharpened image. Three large satellite datasets from urban areas with 4-band spectral resolution (blue, green, red, and near-infrared) were used to evaluate the proposed pansharpening method's performance. The effectiveness of the proposed method is demonstrated by the quantitative and visual results obtained compared to several literature approaches.
Keywords:
Pansharpening
multispectral images
panchromatic images
fusion
autoencoder
deep learning
Journal
IF:
2.6
Papers:
1.2W
Citations:
2.7W