.png){kind=icon}
.png){kind=icon}
Return
Sample-efficient backtrack temporal difference deep reinforcement learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2025.114613.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep reinforcement learning algorithms often require large amounts of training data, particularly in robotic control tasks. To address this limitation, we propose a sample-efficient backtrack temporal difference learning method that enhances target state-action ( Q) value estimation. The proposed method dynamically prioritizes transitions based on their proximity to terminal states using backtrack sampling weights. This prioritization mechanism yields more accurate target Q-values, thereby improving the overall Q-value estimation precision. Furthermore, our analysis uncovers a novel link between curriculum learning and Bellman equation optimization. The proposed method is versatile, applicable to both discrete and continuous action spaces, and readily integrable with off-policy actor-critic algorithms. Extensive experiments show that the proposed method considerably reduces Q-value approximation errors and outperforms baselines across diverse benchmarks, achieving a 28 % performance improvement in four discrete action-space tasks and a 78 % gain in four continuous control tasks.
Journal
K
IF:
7.6
Papers:
1.2W
Citations:
4.5W