.png){kind=icon}
返回
Data Efficient Deep Reinforcement Learning With Action-Ranked Temporal Difference Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TETCI.2024.3369641.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In value-based deep reinforcement learning (RL), value function approximation errors lead to suboptimal policies. Temporal difference (TD) learning is one of the most important methodologies to approximate state-action (Q) value function. In TD learning, it is critical to estimate Q values of greedy actions more accurately because a more accurate target Q value enhances the estimation accuracy of Q value. To improve the estimation accuracy of Q value, we propose an action-ranked TD learning method to enhance the performance of deep RL by weighting each TD error according to the rank of its corresponding state-action pair's value among all the Q values on a state. The proposed method can provide more accurate target values for TD learning, making the estimation of the Q value more accurate. We apply the proposed method to a representative value-based deep RL algorithm, and results show that the proposed method outperforms baselines on 31 out of 40 Atari games. Furthermore, we extend the proposed method to multi-agent deep RL. To adaptively determine the hyperparameter in action-ranked TD learning, we propose a meta action-ranked TD learning. A series of experiments quantitatively verify that our methods outperform baselines on Atari games, StarCraft-II, and Grid World environments.
Keyword:
Reinforcement learning
temporal difference
data efficient
action-rank
meta learning
期刊
I
IF:
6.5
论文数:
1.4K
被引数:
4.5K
机构
引用论文
分享收藏Multi-agent reinforcement learning as a rehearsal for decentralized planning多智能体强化学习作为分散计划的演练.png)
NEUROCOMPUTING
OAAI
NEUROCOMPUTINGIF6.5
OAAI分享收藏Study of bi-directional buck-boost converter topologies for application in electrical vehicle motor drives应用于电动汽车电机驱动的双向buck-boost变换器拓扑研究
PREAI
分享收藏Multi-UAV Mobile Edge Computing and Path Planning Platform Based on Reinforcement Learning基于强化学习的多无人机移动边缘计算与路径规划平台OAAI
OAAI分享收藏
分享收藏A process dissociation framework: Separating automatic from intentional uses of memory过程分离框架: 将自动与有意使用的内存分开
PREAI
分享收藏
查看更多内容