.png){kind=icon}
返回
Sequential action-induced invariant representation for reinforcement learning
.png){kind=icon}
OA
AI
DOI：10.1016/j.neunet.2024.106579.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
How to accurately learn task-relevant state representations from high-dimensional observations with visual distractions is a realistic and challenging problem in visual reinforcement learning. Recently, unsupervised representation learning methods based on bisimulation metrics, contrast, prediction, and reconstruction have shown the ability for task-relevant information extraction. However, due to the lack of appropriate mechanisms for the extraction of task information in the prediction, contrast, and reconstruction-related approaches and the limitations of bisimulation-related methods in domains with sparse rewards, it is still difficult for these methods to be effectively extended to environments with distractions. To alleviate these problems, in the paper, the action sequences, which contain task-intensive signals, are incorporated into representation learning. Specifically, we propose a Sequential Action-induced invariant Representation (SAR) method, which decouples the controlled part (i.e., task-relevant information) and the uncontrolled part (i.e., task-irrelevant information) in noisy observations through sequential actions, thereby extracting effective representations related to decision tasks. To achieve it, the characteristic function of the action sequence's probability distribution is modeled to specifically optimize the state encoder. We conduct extensive experiments on the distracting DeepMind Control suite while achieving the best performance over strong baselines. We also demonstrate the effectiveness of our method at disregarding task-irrelevant information by applying SAR to real-world CARLA-based autonomous driving with natural distractions. Finally, we provide the analysis results of generalization drawn from the generalization decay and t-SNE visualization. Code and demo videos are available at https://github.com/DMUXMU/SAR.git.
Keyword:
Visual reinforcement learning
Representation learning
Action sequence
Visual distraction
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
Study of bi-directional buck-boost converter topologies for application in electrical vehicle motor drives应用于电动汽车电机驱动的双向buck-boost变换器拓扑研究
PREAI
分享收藏Human-level performance in 3D multiplayer games with population-based reinforcement learning基于人口的强化学习在3D多人游戏中的人类水平表现.png)
SCIENCEOAAI
SCIENCEIF45.8
OAAI分享收藏Identifying the key frames: An attention-aware sampling method for action recognitionPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏Unified curiosity-Driven learning with smoothed intrinsic reward estimationPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏 没有更多内容
没有更多内容