.png){kind=icon}
.png){kind=icon}
Return
Combined data augmentation framework for generalizing deep reinforcement learning from pixels
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2024.125810.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The inherent generalization problem in reinforcement learning (RL) is exacerbated when dealing with highdimensional visual input. Data augmentation is a promising method to improve sample efficiency and generalization capability in RL. However, due to the sensitivity of RL training, the plain use of data augmentation can introduce more variability into the training process, making the optimization process more complex, leading to reduced sample efficiency, unstable training, and further deterioration of generalization performance. We propose a Combined Data Augmentation (CDA) framework that applies all data augmentation operators to visual RL. The CDA framework divides data augmentation methods into three data streams: no augmentation, pixel-level augmentation, and spatial-level augmentation. By treating each data stream separately, the CDA framework provides the visual RL pipeline with slight hardness and rich diversity data augmentation methods. It introduces different inductive biases for the agent, which can improve the sample efficiency while satisfying the smoothness assumption, i.e., ensuring that small changes in input data lead to small changes in model predictions, and enhancing the generalization performance, thus extending the highperformance policy to new environments that have not been seen before. The CDA framework is extensively evaluated on DeepMind Control Suite and corresponding generalization tasks. Experimental results show that CDA achieves better or comparable asymptotic performance in all training environments, and exhibits higher sample efficiency. Additionally, CDA outperforms other baseline methods in terms of generalization performance on 6 out of 10 tasks.
Keywords:
Reinforcement learning
Data augmentation
Generalization
Sample efficiency
Journal
IF:
7.5
Papers:
2.9W
Citations:
10.2W