logo
科言猫学术研究的AI总结
language
手机端手机端
  • Phone Feature微信小程序
简体中文简体中文
notice消息通知
博客博客
delete
arrow
返回

Memory-Driven Querying Transformer for Vision-and-Language Navigation

delete2026-01-01
delete0
PRE
AI
J
Jin, Chuan
Z
Zhang, Yuting
Y
Yang, Boyuan *
DOI：10.1007/978-981-95-5740-0_21delete
delete原文链接
delete原文求助
delete分享
delete收藏
摘要

摘要

En 中文
视觉与语言导航（VLN）是指机器人通过遵循人类自然语言指令在视觉环境中导航的过程。VLN的一个关键挑战，特别是在长距离导航中，是如何高效地维持机器人对历史导航步骤的记忆。为解决此问题，我们提出了记忆驱动查询Transformer（MDQT），这是一种新型模型，利用可学习的固定长度记忆向量来编码历史信息。在每一步，记忆向量通过查询Transformer（Q-Former）与语言和视觉信息进行交互。MDQT随后预测下一步动作，并使用记忆更新模块（MUM）更新记忆向量。该模型在三个任务上进行预训练，并在模拟环境中进行微调。实验结果表明，MDQT在Room-to-Room（R2R）数据集上达到了53%的成功率（SR），优于基线模型。消融研究进一步验证了我们的预训练任务和MUM在提升导航性能方面的有效性。此外，我们将模型从模拟环境迁移到物理机器人上，使其能够在现实环境中导航。
Keyword:
Vision-and-language navigation
Robot learning
Embodied AI

期刊

P
PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2025, PT XI
IF:
0
论文数:
29
被引数:
0

机构

N
nanjing university
学者数:
7.8W
论文数: 5.6W
被引数: 87
N
nankai university
学者数:
4.8W
论文数: 3.3W
被引数: 74
引用论文

引用论文

Multiple Visual Features in Topological Map for Vision-and-Language Navigation
err2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
IF0
err2024-10-14
err0
PREAI
errLiu,Ruonan; Kong,Ping; Zhang,Weidong
err分享
err收藏
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training
err2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
IF0
err2020-06-01
err0
errOAAI
errWeituo Hao; Chunyuan Li; Xiujun Li; Lawrence Carin; Jianfeng Gao
err分享
err收藏
Long Short-Term Memory长短期记忆
errNeural Computation
IF0
err1997-11-01
err0
PREAI
errSepp Hochreiter; Jürgen Schmidhuber
err分享
err收藏
Robust Navigation with Language Pretraining and Stochastic Sampling
errProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
IF0
err2019-01-01
err0
errOAAI
errXiujun Li; Chunyuan Li; Qiaolin Xia; Yonatan Bisk; Asli Celikyilmaz; Jianfeng Gao; Noah A. Smith; Yejin Choi
err分享
err收藏
NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models
errProceedings of the AAAI Conference on Artificial Intelligence
IF0
err2024-03-24
err0
errOAAI
errGengze Zhou; Yicong Hong; Qi Wu
err分享
err收藏
Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
err2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
IF0
err2019-06-01
err0
errOAAI
errXin Wang; Qiuyuan Huang; Asli Celikyilmaz; Jianfeng Gao; Dinghan Shen; Yuan-Fang Wang; William Yang Wang; Lei Zhang
err分享
err收藏
GLM: General Language Model Pretraining with Autoregressive Blank InfillingGLM: 具有自回归空白填充的通用语言模型预训练
errProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
IF0
err2022-01-01
err0
errOAAI
errZhengxiao Du; Yujie Qian; Xiao Liu; Ming Ding; Jiezhong Qiu; Zhilin Yang; Jie Tang
err分享
err收藏
Improved VLN-BERT with Reinforcing Endpoint Alignment for Vision-and-Language Navigation
errCommunications in Computer and Information Science
IF0
err2024-01-01
err0
PREAI
errJin,Chuan; Yang,Boyuan; Liu,Ruonan
err分享
err收藏
DAP: Domain-Aware Prompt Learning for Vision-and-Language Navigation
errICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
IF0
err2024-04-14
err0
PREAI
errLiu,Ting; Hu,Yue; Wu,Wansen; Wang,Youkai; Xu,Kai; Yin,Quanjun
err分享
err收藏
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
err2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
IF0
err2020-06-01
err0
errOAAI
errYuankai Qi; Qi Wu; Peter Anderson; Xin Wang; William Yang Wang; Chunhua Shen; Anton van den Hengel
err分享
err收藏
学者 查看更多内容