.png){kind=icon}
返回
Solving time-delay issues in reinforcement learning via transformers
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-024-05830-2.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The presence of observation and action delays in remote control scenarios significantly challenges the decision-making of agents that depend on immediate interactions, particularly within traditional deep reinforcement learning (DRL) algorithms. Existing approaches attempt to tackle this problem through various strategies, such as predicting delayed states, transforming delayed Markov Decision Processes (MDPs) into delay-free equivalents. However, both model-free and model-based methods require extensive online data, making them time-consuming and resource-intensive. To effectively handle time-delay challenges and develop a competent and robust RL algorithm, the Augmented Decision Transformer (ADT) is proposed as the first offline RL algorithm designed to enable agents to manage diverse tasks with various constant delays. It transforms a deterministic delayed MDP (DDMDP) into a standard MDP by simulating trajectories in delayed environments using offline dataset from undelayed environments. The Decision Transformer, an autoregressive model, is then employed to train a decision model based on expected rewards, past state sequences and past action sequences. Extensive experiments conducted on MuJoCo and Adroit tasks validate the robustness and efficiency of the ADT, with its average performance across all tasks being 56% better than the worst-performing comparative algorithms. The results demonstrate that the ADT can outperform state-of-the-art RL counterparts, achieving superior performance across various tasks with different delay conditions.
Keyword:
Deep reinforcement learning
Time delay
Deterministic delayed Markov Decision Process
Offline reinforcement learning
Decision transformer
期刊
IF:
3.5
论文数:
7.6K
被引数:
1.7W
机构
引用论文
.png){kind=icon}
分享收藏分享收藏
分享收藏Remote human-robot collaboration: A cyber-physical system application for hazard manufacturing environment远程人机协作: 用于危险制造环境的网络物理系统应用程序
PREAI
分享收藏
查看更多内容