Learning controllable elements oriented representations for reinforcement learning

doi:10.1016/j.neucom.2023.126455

Learning controllable elements oriented representations for reinforcement learning

2023-09-01

4

PRE

AI

DOI：10.1016/j.neucom.2023.126455

Original

Original request for help

Save

Abstract

En 中文

Deep Reinforcement Learning (deep RL) has been successfully applied to solve various decision-making problems in recent years. However, the observations in many real-world tasks are often high dimensional and include much task-irrelevant information, limiting the applications of RL algorithms. To tackle this problem, we propose LCER, a representation learning method that aims to provide RL algorithms with compact and sufficient descriptions of the original observations. Specifically, LCER trains representations to retain the controllable elements of the environment, which can reflect the action-related environment dynamics and thus are likely to be task-relevant. We demonstrate the strength of LCER on the DMControl Suite, proving that it can achieve state-of-the-art performance. LCER enables the pixel -based SAC to outperform state-based SAC on the DMControl 100 K benchmark, showing that the obtained representations can match the oracle descriptions (i.e. the physical states) of the environment. We also carry out experiments to show that LCER can efficiently filter out various distractions, especially when those distractions are not controllable.& COPY; 2023 Elsevier B.V. All rights reserved.

Keywords:

Reinforcement learning

Representation learning

Learning controllable elements oriented representations for reinforcement learning

Abstract

Journal

Organization