Combining Model-Based Q-Learning With Structural Knowledge Transfer for Robot Skill Learning

Zhen Deng

doi:10.1109/TCDS.2017.2718938

Return

Combining Model-Based Q-Learning With Structural Knowledge Transfer for Robot Skill Learning

2019-03-01

PRE

DOI：10.1109/TCDS.2017.2718938

Original

Original request for help

Save

Abstract

En 中文

Learning skills autonomously is a particularly important ability for an autonomous robot. A promising approach is reinforcement learning (RL) where agents learn policy through interaction with its environment. One problem of RL algorithm is how to tradeoff the exploration and exploitation. Moreover, multiple tasks also make a great challenge to robot learning. In this paper, to enhance the performance of RL, a novel learning framework integrating RL with knowledge transfer is proposed. Three basic components are included: 1) probability policy reuse; 2) dynamic model learning; and 3) model-based Q-learning. In this framework, the prelearned skills are leveraged for policy reuse and dynamic learning. In model-based Q-learning, the Gaussian process regression is used to approximate the Q-value function so as to suit for robot control. The prior knowledge retrieved from knowledge transfer is integrated into the model-based Q-learning to reduce the needed learning time. Finally, a human-robot handover experiment is performed to evaluate the learning performance of this learning framework. Experiment results show that fewer exploration is needed to obtain a high expected reward, due to the prior knowledge obtained from knowledge transfer.

Keywords:

Gaussian process (GP)

knowledge transfer

model-based Q-learning

reinforcement learning (RL)

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.