Physics-informed residual reinforcement learning via expert prior knowledge for safe and efficient autonomous merging

Dequan Zeng

doi:10.1016/j.neucom.2026.134631

Physics-informed residual reinforcement learning via expert prior knowledge for safe and efficient autonomous merging

2026-07-28

0

PRE

AI

D

Dequan Zeng

Z

Zhishao Ni

Z

Zhuoren Li *

X

Xinrui Zhang

M

Ming Liu

Y

Yiming Hu

C

Chen Sun

B

Bo Leng

DOI：10.1016/j.neucom.2026.134631

Original

Original request for help

Save

Abstract

En 中文

Autonomous on-ramp merging is a safety-critical and highly interactive driving task that requires reliable feasibility, safety assurance, and timely maneuver completion. However, existing reinforcement learning (RL) approaches often suffer from low sample efficiency in such high-dimensional and highly dynamic environments, and pure reward-driven exploration struggles to accommodate driving safety and efficiency objectives, leading to either overly aggressive or overly conservative driving behaviors. To address this challenge, we propose PR-EPK, a physics-informed residual RL framework that integrates model-based and analytical expert knowledge into both policy learning and execution. An MPC-based prior provides dynamically feasible baseline controls, while the RL agent learns bounded residual corrections to refine expert behavior, improving exploration efficiency and convergence stability. A risk-adaptive hybrid execution mechanism further regulates residual authority online as a soft safety heuristic during execution. In addition, a decoupled dual-critic constrained learning scheme separates task-return and safety-cost estimation to mitigate objective coupling, and a progressive curriculum strategy gradually transitions from expert-guided training to autonomous policy enhancement. Simulation results across varying traffic densities demonstrate that PR-EPK achieves faster and more stable convergence, consistently higher success rates, significantly reduced collision rates and shorter average merge times compared with representative baselines. These results validate the effectiveness of the proposed framework and demonstrate that structured integration of expert knowledge within residual RL provides a practical pathway toward safe and efficient autonomous merging in highly dynamic traffic.

Physics-informed residual reinforcement learning via expert prior knowledge for safe and efficient autonomous merging

Abstract

Journal

Organization