Self-Referencing Agents for Unsupervised Reinforcement Learning

Zhao, Andrew

doi:10.1016/j.neunet.2025.107448

Self-Referencing Agents for Unsupervised Reinforcement Learning

2025-05-27

0

PRE

AI

DOI：10.1016/j.neunet.2025.107448

Original

Original request for help

Save

Abstract

En 中文

Current unsupervised reinforcement learning methods often overlook reward nonstationarity during pre-training and the forgetting of exploratory behavior during fine-tuning. Our study introduces Self-Reference (SR), a novel add-on module designed to address both issues. SR stabilizes intrinsic rewards through historical referencing in pre-training, mitigating nonstationarity. During fine-tuning, it preserves exploratory behaviors, retaining valuable skills. Our approach significantly boosts the performance and sample efficiency of existing URL model-free methods on the Unsupervised Reinforcement Learning Benchmark, improving IQM by up to 17% and reducing the Optimality Gap by 31%. This highlights the general applicability and compatibility of our add-on module with existing methods.

Keywords:

Reinforcement learning

Unsupervised reinforcement learning

Pretraining

Finetuning

Self-Referencing Agents for Unsupervised Reinforcement Learning

Abstract

Journal

Organization