State Abstraction via Deep Supervised Hash Learning

doi:10.1109/TNNLS.2024.3467338

State Abstraction via Deep Supervised Hash Learning

2024-01-01

0

PRE

AI

DOI：10.1109/TNNLS.2024.3467338

Original

Original request for help

Save

Abstract

En 中文

State abstraction is a widely used technique in reinforcement learning (RL) that compresses the state space to accelerate learning algorithms. However, designing an effective abstraction function in large-scale or high-dimensional state space problems remains a significant challenge. In this brief, we present a novel state abstraction method based on deep supervised hash learning (DSH) and provide a theoretical analysis of its near-optimal property. Furthermore, by leveraging the DSH-based representation as the optimization objective, we propose a direct and concise optimization method based on the target value. In addition, we construct an auxiliary learning task for state abstraction that can be combined with various RL algorithms. In particular, we apply the DSH-based state abstraction to both deep Q-learning (DQN) and soft actor-critic (SAC). Extensive experiments are conducted on Atari and several classic control benchmarks to evaluate the effectiveness of the DSH-based state abstraction method, showing that our method surpasses existing state abstraction algorithms in performance.

Keywords:

Vectors

Technological innovation

Stationary state

Software

Measurement

Learning systems

Computational modeling

Binary codes

Benchmark testing

Topology

Deep supervised hash learning (DSH)

near-optimal property

reinforcement learning (RL)

state abstraction

State Abstraction via Deep Supervised Hash Learning

Abstract

Journal

Organization