Stochastic Integrated ActorCritic for Deep Reinforcement Learning

doi:10.1109/TNNLS.2022.3212273

Stochastic Integrated ActorCritic for Deep Reinforcement Learning

2024-05-01

4

PRE

AI

DOI：10.1109/TNNLS.2022.3212273

Original

Original request for help

Save

Abstract

En 中文

We propose a deep stochastic actor-critic algorithm with an integrated network architecture and fewer parameters. We address stabilization of the learning procedure via an adaptive objective to the critic's loss and a smaller learning rate for the shared parameters between the actor and the critic. Moreover, we propose a mixed on-off policy exploration strategy to speed up learning. Experiments illustrate that our algorithm reduces the sample complexity by 50%-93% compared with the state-of-the-art deep reinforcement learning (RL) algorithms twin delayed deep deterministic policy gradient (TD3), soft actor-critic (SAC), proximal policy optimization (PPO), advantage actor-critic (A2C), and interpolated policy gradient (IPG) over continuous control tasks LunarLander, BipedalWalker, BipedalWalkerHardCore, Ant, and Minitaur in the OpenAI Gym.

Keywords:

Training

Task analysis

Complexity theory

Linear programming

Network architecture

Decoding

Tensors

Actor-critic

adaptive objective

deep reinforcement learning (RL)

integrated network

mixed on-off policy exploration

sample complexity

Stochastic Integrated ActorCritic for Deep Reinforcement Learning

Abstract

Journal

Organization