.png){kind=icon}
.png){kind=icon}
Return
Light field angular super-resolution by view-specific queries
.png){kind=icon}
PRE
AI
DOI：10.1007/s00371-024-03620-y.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Light field angular super-resolution (LFASR) aims to reconstruct the densely sampled light field from sparsely sampled inputs. Recently, convolutional neural network-based methods have achieved encouraging results. However, most of these approaches use view-specific characteristics of the target dense light (i.e., contents and view locations) separately, the geometry structure information of the target light field is not fully explored. To this end, we propose view-specific queries to integrate the view location information of the dense light field into Transformer (dubbed as ViewFormer) for LFASR. In particular, we first leverage a Transformer encoder to process the input sparsely sampled light field. Then, the view interpolation operation is used to process the extracted subaperture features along horizontal and vertical directions of the target light field, generating the new sub-aperture representations dubbed as view-specific queries. Next, the view-specific queries contain the view coordinate information of the target light field, and are dynamically enhanced by a Transformer decoder layer by layer. The enhanced view-specific queries are fed to the reconstruction module for final light field synthesis. Additionally, to further mine more information on input sparsely sampled light field, we employ the channel attention scheme to the building blocks of the Transformer. Extensive experiments are performed on commonly-used LFASR benchmarks. ViewFormer achieves new state-of-the-art results compared with other methods on popular LFASR benchmarks, including real-world and synthetic data.
Keywords:
Light field angular super-resolution
Transformer
Visual attention
View synthesis
Journal
IF:
2.9
Papers:
4.5K
Citations:
6.5K