.png){kind=icon}
Return
Model-Based Transfer Reinforcement Learning Based on Graphical Model Representations
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2021.3107375.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Reinforcement learning (RL) plays an essential role in the field of artificial intelligence but suffers from data inefficiency and model-shift issues. One possible solution to deal with such issues is to exploit transfer learning. However, interpretability problems and negative transfer may occur without explainable models. In this article, we define Relation Transfer as explainable and transferable learning based on graphical model representations, inferring the skeleton and relations among variables in a causal view and generalizing to the target domain. The proposed algorithm consists of the following three steps. First, we leverage a suitable casual discovery method to identify the causal graph based on the augmented source domain data. After that, we make inferences on the target model based on the prior causal knowledge. Finally, offline RL training on the target model is utilized as prior knowledge to improve the policy training in the target domain. The proposed method can answer the question of what to transfer and realize zero-shot transfer across related domains in a principled way. To demonstrate the robustness of the proposed framework, we conduct experiments on four classical control problems as well as one simulation to the real-world application. Experimental results on both continuous and discrete cases demonstrate the efficacy of the proposed method.
Keywords:
Task analysis
Data models
Transfer learning
Sun
Graphical models
Training
Reinforcement learning
Auxiliary variable
causal graph
reinforcement learning (RL)
transfer learning
Journal
IF:
8.9
Papers:
7.6K
Citations:
7.2W
Organization
Cited Papers
Adherence of sickle erythrocytes to vascular endothelial cells: requirement for both cell membrane changes and plasma factors.png)
Blood
OAAI
BloodIF0
OAAIShareSaveReinforcement Learning Tracking Control for Robotic Manipulator With Kernel-Based Dynamic Model
PREAI
ShareSaveShareSaveShareSaveShareSave
View more