Spatial-temporal feature refine network for single image super-resolution

Jiayi Qin

doi:10.1007/s10489-022-03741-8

2022-08-10

PRE

DOI：10.1007/s10489-022-03741-8

原文链接

原文求助

摘要

En 中文

In recent years, numerous convolution neural networks (CNNs) for single image super-resolution (SISR) have shown powerful capability in image reconstruction. Especially, accurate and compact networks receive widespread attention due to their superiority in running speed on resources-limited devices. However, in most lightweight CNN-based methods, the limitation of informative features extraction results in mediocre reconstructed images. In this paper, we propose a spatial-temporal feature refine network (STFRN) to alleviate the above problem by extract features from diverse dimensions. Specifically, we first introduce a spatial-temporal stage learning (STSL) of our STFRN in two distinct views: 1) for temporal feature extraction, we enhance the relevance of various stages and introduce a persistent memory via multi-stage learning, thereby boosting its reconstruction capability; 2) for spatial feature extraction, we enlarge the receptive fields by densely deepening the network to capture more context features. In the time dimension, we only maintain the main stage, i.e., the last stage, for efficient training. In addition to STSL, we elaborately design an effective multi-attention enhanced residual block (MERB), which further refines informative features at the depth level. In detail, by combining multiple attention blocks, we achieve the complementary features from diverse views (i.e., in point-wise, channel-wise, and spatial dimensions). Compared with ordinary residual blocks, MERB realizes better results while consuming less computational resources. Like most literature, we also utilize residual learning and dense connection to promote performance. Extensive experiments show our STFRN levering on the STSL and MERBs is superior to other state-of-the-art methods in quality and quantity.

Keyword:

Single image super-resolution

Lightweight convolution neural network

Spatial-temporal feature refine network

Multi-attention enhanced residual block