.png){kind=icon}
.png){kind=icon}
Return
Multi-Scale Spatial-Angular Collaborative Guidance Network for Heterogeneous Light Field Spatial Super-Resolution
.png){kind=icon}
PRE
AI
DOI：10.1109/TBC.2024.3420748.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Light Field (LF) imaging captures the spatial and angular information of light rays in the real world and enables various applications, including digital refocusing and single-shot depth estimation. Unfortunately, due to the limited sensor size of LF cameras, the captured LF images suffer from low spatial resolution while providing a dense angular sampling. Existing single-input LF spatial super-resolution (SR) methods usually utilize the inherent sub-pixel information to recover high-frequency textures, but they struggle in large-scale SR tasks (e.g., 8 $\times$ ). Conversely, the heterogeneous imaging approach combining an LF camera and a 2D digital camera can capture richer information for effective large-scale reconstruction. To this end, this paper proposes a multi-scale spatial-angular collaborative guidance network (LF-MSACGNet) for heterogeneous LF spatial SR. Specifically, a context-guided deformable alignment module is first designed, which utilizes high-level feature information to achieve precise alignment between the low-resolution LF image and the 2D high-resolution image. Subsequently, a Transformer-driven spatial-angular collaborative guidance module is constructed to explore the spatial-angular correlation and complementarity. This allows for an effective fusion of the multi-resolution spatial-angular features. Finally, the SR LF image is reconstructed through a spatial-angular aggregation module. In addition, a multi-scale training strategy is adopted to subdivide the challenging large-scale SR task into multiple simple tasks to boost the SR performance. Experimental results on seven public datasets show that the proposed method outperforms the state-of-the-art SR methods in both quantitative and qualitative comparison, and exhibits favorable robustness to wide baseline LF images.
Keywords:
Imaging
Task analysis
Image reconstruction
Correlation
Collaboration
Transformers
Spatial resolution
Light field
spatial super-resolution
heterogeneous imaging
transformer
spatial-angular correlation
Journal
IF:
4.8
Papers:
2.1K
Citations:
3.0K