Iterative Back Projection Network Based on Deformable 3D Convolution

Chengzhi Luo

doi:10.1109/ACCESS.2023.3325659

Return

Iterative Back Projection Network Based on Deformable 3D Convolution

2023-01-01

Chengzhi Luo

Bing Li *

Feng Liu

DOI：10.1109/ACCESS.2023.3325659

Original

Original request for help

Save

Abstract

En 中文

Video super-resolution technology enhances the display quality of videos by obtaining high-resolution videos from low-resolution videos. Unlike single-image super-resolution, utilizing information between adjacent video frames is crucial in video super-resolution. To improve the performance of video super-resolution reconstruction, a model combining deformable 3D convolution and iterative back projection is proposed to fully exploit the temporal-spatial correlation of video frames. The model takes multiple consecutive video frames as input and outputs the super-resolution reconstruction of the middle frame, including three modules: multi-scale feature extraction, feature fusion, and high-resolution reconstruction. Firstly, multi-scale 3D convolution is used for preliminary feature extraction. Then, deformable 3D convolution and iterative back projection are combined for feature fusion. Finally, multiple residual dense blocks and sub-pixel convolution are used for high-resolution reconstruction, and global residual connections are utilized to obtain the reconstructed high-resolution video. Experimental results on the Vid4 dataset demonstrate that compared to existing methods, this method can effectively improve the peak signal-to-noise ratio and structural similarity performance and achieve better visual effects with 4x super-resolution magnification.

Keywords:

Superresolution

Image reconstruction

Feature extraction

Convolution

Three-dimensional displays

Image resolution

Optical flow

Video super-resolution

adjacent video frames

deformable 3D convolution

iterative back-projection