.png){kind=icon}
.png){kind=icon}
Return
State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TMC.2025.3601531.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing white-box adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and state-specific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the Adversarial Victim Dynamics Markov Decision Process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose the Selective State-Aware Reinforcement adversarial attack (STAR), to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks.
Keywords:
Perturbation methods
Robots
Glass box
Stars
Robustness
Deep reinforcement learning
Navigation
Trajectory
Training
Robot sensing systems
Markov decision process
deep reinforcement learning
adversarial attack
robotic manipulation
Journal
IF:
9.2
Papers:
5.6K
Citations:
1.8W
Organization
View more organizations