.png){kind=icon}
Return
Active Perception for Visual-Language Navigation
.png){kind=icon}
PRE
AI
DOI：10.1007/s11263-022-01721-6.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Visual-language navigation (VLN) is the task of entailing an agent to carry out navigational instructions inside photo-realistic environments. One of the key challenges in VLN is how to conduct robust navigation by mitigating the uncertainty caused by ambiguous instructions and insufficient observation of the environment. Agents trained by current approaches typically suffer from this and would consequently struggle to take navigation actions at every step. In contrast, when humans face such a challenge, we can still maintain robust navigation by actively exploring the surroundings to gather more information and thus make a more confident navigation decision. This work draws inspiration from human navigation behavior and endows an agent with an active perception ability for more intelligent navigation. To achieve this, we propose an end-to-end framework for learning an exploration policy that decides (i) when and where to explore, (ii) what information is worth gathering during exploration, and (iii) how to adjust the navigation decision after the exploration. In this way, the agent is able to turn its past experiences as well as new explored knowledge to contexts for more confident navigation decision making. In addition, an external memory is used to explicitly store the visited visual environments and thus allows the agent to adopt a late action-taking strategy to avoid duplicate exploration and navigation movements. Our experimental results on two standard benchmark datasets show promising exploration strategies emerged from training, which leads to significant boost in navigation performance.
Keywords:
Visual-language navigation
Active perception
Curriculum reinforcement learning
Journal
IF:
9.3
Papers:
3.9K
Citations:
2.8W
Organization
Cited Papers
ShareSaveEffect of Lizards on Spider Populations: Manipulative Reconstruction of a Natural Experiment.png)
Science
ScienceIF0
PREAI
ShareSaveImpact of antibiotic treatment on immunotherapy response in women with recurrent gynecologic cancer
PREAI
ShareSaveShareSave
ShareSaveInternational financial reporting standards and experts’ perceptions of disclosure qualityAbacus
OAAI
AbacusIF0
OAAIShareSaveShareSave
View more