.png){kind=icon}
.png){kind=icon}
Return
Texture-Enhanced Light Field Super-Resolution With Spatio-Angular Decomposition Kernels
.png){kind=icon}
OA
AI
DOI：10.1109/TIM.2022.3152242.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Despite the recent progress in light field super-resolution (LFSR) achieved by convolutional neural networks, the correlation information of LF images has not been sufficiently studied and exploited due to the complexity of 4-D LF data. To cope with such high-dimensional LF data, most of the existing LFSR methods resorted to decomposing it into lower dimensions and subsequently performing optimization on the decomposed subspaces. However, these methods are inherently limited as they neglected the characteristics of the decomposition operations and only utilized a limited set of LF subspaces ending up failing to sufficiently extract spatio-angular features and leading to a performance bottleneck. To overcome these limitations, in this article, we comprehensively discover the potentials of LF decomposition and propose a novel concept of decomposition kernels. In particular, we systematically unify the decomposition operations of various subspaces into a series of such decomposition kernels, which are incorporated into our proposed decomposition kernel network (DKNet) for comprehensive spatio-angular feature extraction. The proposed DKNet is experimentally verified to achieve considerable improvements compared with state-of-the-art methods. To further improve DKNet in producing more visually pleasing LFSR results, based on the VGG network, we propose a light field VGG (LFVGG) loss to guide the texture-enhanced DKNet (TE-DKNet) to generate rich authentic textures and enhance LF images' visual quality significantly. We also propose an indirect evaluation metric by taking advantage of LF material recognition to objectively assess the perceptual enhancement brought by the LFVGG loss.
Keywords:
Feature extraction
Kernel
Cameras
Spatial resolution
Light fields
Visualization
Synthetic aperture sonar
Convolutional neural network (CNN)
deep learning
image processing
light field (LF) imaging
super-resolution
Journal
IF:
5.9
Papers:
1.9W
Citations:
5.8W