.png){kind=icon}
Return
Reinforcement Learning Control With Knowledge Shaping
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2023.3243631.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We aim at creating a transfer reinforcement learning framework that allows the design of learning controllers to leverage prior knowledge extracted from previously learned tasks and previous data to improve the learning performance of new tasks. Toward this goal, we formalize knowledge transfer by expressing knowledge in the value function in our problem construct, which is referred to as reinforcement learning with knowledge shaping (RL-KS). Unlike most transfer learning studies that are empirical in nature, our results include not only simulation verifications but also an analysis of algorithm convergence and solution optimality. Also different from the well-established potential-based reward shaping methods which are built on proofs of policy invariance, our RL-KS approach allows us to advance toward a new theoretical result on positive knowledge transfer. Furthermore, our contributions include two principled ways that cover a range of realization schemes to represent prior knowledge in RL-KS. We provide extensive and systematic evaluations of the proposed RL-KS method. The evaluation environments not only include classical RL benchmark problems but also include a challenging task of real-time control of a robotic lower limb with a human user in the loop.
Keywords:
Task analysis
Knowledge transfer
Reinforcement learning
Transfer learning
Silicon
Knowledge representation
Convergence
Reinforcement learning (RL)
reward shaping
transfer learning
value function
Journal
IF:
8.9
Papers:
7.6K
Citations:
7.2W
Organization
Cited Papers
ShareSaveShareSaveOptimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming.png)
AUTOMATICA
AUTOMATICAIF5.9
PREAI
ShareSave
ShareSave
View more