.png){kind=icon}
返回
Transformer-Based Self-Supervised Monocular Depth and Visual Odometry
.png){kind=icon}
PRE
AI
DOI：10.1109/JSEN.2022.3227017.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Self-supervised monocular depth and visual odometry (VO) are often cast as coupled tasks. Accurate depth contributes to precise pose estimation and vice versa. Existing architectures typically exploit stacking convolution layers and long short-term memory (LSTM) units to capture long-range dependencies. However, their intrinsic locality hinders the model from getting the expected performance gain. In this article, we propose a Transformer-based architecture, named Transformer-based self-supervised monocular depth and VO (TSSM-VO), to tackle these problems. It comprises two main components: 1) a depth generator that leverages the powerful capability of multihead self-attention (MHSA) on modeling long-range spatial dependencies and 2) a pose estimator built upon a Transformer to learn long-range temporal correlations of image sequences. Moreover, a new data augmentation loss based on structural similarity (SSIM) is introduced to constrain further the structural similarity between the augmented depth and the augmented predicted depth. Rigorous ablation studies and exhaustive performance comparison on the KITTI and Make3D datasets demonstrate the superiority of TSSM-VO over other self-supervised methods. We expect that TSSM-VO would enhance the ability of intelligent agents to understand the surrounding environments.
Keyword:
Data augmentation loss
long-range dependencies
monocular depth estimation
multihead self-attention (MHSA)
visual odometry (VO)
期刊
IF:
4.5
论文数:
2.1W
被引数:
7.3W
机构
引用论文
Meta-analyses of IL1A polymorphisms and the risk of several autoimmune diseases published in databases.png)
PLOS ONE
OAAI
PLOS ONEIF0
OAAI分享收藏Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
PREAI
分享收藏
分享收藏
查看更多内容