.png){kind=icon}
返回
Sampling Efficient Deep Reinforcement Learning Through Preference-Guided Stochastic Exploration
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2023.3317628.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Stochastic exploration is the key to the success of the deep Q-network (DQN) algorithm. However, most existing stochastic exploration approaches either explore actions heuristically regardless of their Q values or couple the sampling with Q values, which inevitably introduce bias into the learning process. In this article, we propose a novel preference-guided epsilon-greedy exploration algorithm that can efficiently facilitate exploration for DQN without introducing additional bias. Specifically, we design a dual architecture consisting of two branches, one of which is a copy of DQN, namely, the Q branch. The other branch, which we call the preference branch, learns the action preference that the DQN implicitly follows. We theoretically prove that the policy improvement theorem holds for the preference-guided epsilon-greedy policy and experimentally show that the inferred action preference distribution aligns with the landscape of corresponding Q values. Intuitively, the preference-guided epsilon-greedy exploration motivates the DQN agent to take diverse actions, so that actions with larger Q values can be sampled more frequently, and those with smaller Q values still have a chance to be explored, thus encouraging the exploration. We comprehensively evaluate the proposed method by benchmarking it with well-known DQN variants in nine different environments. Extensive results confirm the superiority of our proposed method in terms of performance and convergence speed.
Keyword:
Data efficiency
deep Q network (DQN)
deep reinforcement learning (DRL)
preference-guided exploration
stochastic policy
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
.png){kind=icon}
Efficient Deep Reinforcement Learning With Imitative Expert Priors for Autonomous Driving具有模拟专家先验的高效深度强化学习，用于自动驾驶OAAI
OAAI分享收藏Study of bi-directional buck-boost converter topologies for application in electrical vehicle motor drives应用于电动汽车电机驱动的双向buck-boost变换器拓扑研究
PREAI
分享收藏
查看更多内容