.png){kind=icon}
.png){kind=icon}
Return
A transformer-CNN parallel network for image guided depth completion
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.110305.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Image guided depth completion aims to predict a dense depth map from sparse depth measurements and the corresponding single color image. However, most state-of-the-art methods only rely on convolutional neural network (CNN) or transformer. In this paper, we propose a transformer -CNN parallel network (TCPNet) to integrate the advantages of CNN in local detail recovery and transformer in long-range semantic modeling. Specifically, our CNN branch adopts dense connection to strengthen feature propagation. Since the common transformer computes self -attention based on all the tokens in the window, no matter if they are relevant or not, this will inevitably introduce interferences and noises. To improve the self -attention accuracy, we propose a correlation -based transformer to only allow nearest neighbor tokens to participate in the self -attention computation. We also design a multi -scale conditional random field (CRF) module to implement multi -scale high -dimensional filtering for depth refinement. The comprehensive experimental results on KITTI and NYUv2 demonstrate that our method outperforms the state-of-the-art methods.
Keywords:
Depth completion
Convolutional neural network
Transformer
Token correlation
Conditional random field
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W