Sparse Variational Deterministic Policy Gradient for Continuous Real-Time Control

Jongchan Baek

doi:10.1109/TIE.2020.3021607

Return

Sparse Variational Deterministic Policy Gradient for Continuous Real-Time Control

2021-10-01

PRE

DOI：10.1109/TIE.2020.3021607

Original

Original request for help

Save

Abstract

En 中文

Recent advancements in deep reinforcement learning for real control tasks have received interest from many researchers and field engineers in a variety of industrial areas. However, in most cases, optimal policies obtained by deep reinforcement learning are difficult to implement on cost-effective and lightweight platforms such as mobile devices. This can be attributed to their computational complexity and excessive memory usage. For this reason, this article proposes an off-policy deep reinforcement learning algorithm called the sparse variational deterministic policy gradient (SVDPG). SVDPG provides highly efficient policy network compression under the standard reinforcement learning framework. The proposed SVDPG integrates Bayesian pruning, which is known as a state-of-the-art neural network compression technique, with the policy update in an actor-critic architecture for reinforcement learning. It is demonstrated that SVDPG achieves a high compression rate of policy networks for continuous control benchmark tasks while preserving a competitive performance. The superiority of SVDPG in low-computing power devices is proven by comparing the level of compression in terms of the memory requirements and computation time on a commercial microcontroller unit. Finally, it is confirmed that the proposed SVDPG is also reliable in real-world scenarios since it can be applied to the swing-up control of an inverted pendulum system.

Keywords:

Bayes methods

Machine learning

Neural networks

Learning (artificial intelligence)

Computational modeling

Standards

Optimization

Bayesian compression

deep reinforcement learning

inverted pendulum system

sparse Bayesian deep learning

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.