.png){kind=icon}
.png){kind=icon}
Return
Digital twin-enabled nested Q-learning for multi-layer production planning and inventory control
.png){kind=icon}
PRE
AI
L
K
DOI：10.1016/j.ijpe.2026.109979.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Modern production systems face increasing uncertainty due to demand variability, supply delays, and machine breakdowns. Traditional hierarchical Production Planning and Inventory Control (PPIC) approaches often lack the adaptability required to manage such volatility across multiple planning horizons. This study proposes a digital twin-enabled nested Q-learning framework for coordinating strategic and operational decisions across monthly, weekly, and daily levels in a flexible job shop environment. The digital twin acts as a real-time simulation layer, capturing dynamic system states such as inventory levels, machine availability, and disruption events. A hierarchical reinforcement learning (HRL) structure, based on nested Q-learning agents, is used to learn adaptive policies for production planning, material procurement, and job scheduling. Computational experiments under stochastic conditions show that the proposed approach achieves cost reductions of 4%-15%, fill rates up to 99%, and improved machine utilization, outperforming conventional rule-based strategies. By integrating real-time feedback with multi-layer decision-making, this framework contributes to the development of autonomous, resilient, and Industry 4.0-ready production systems.
Keywords:
Digital twin
Hierarchical reinforcement learning
Production Planning and Inventory Control
(PPIC)
Flexible job shop scheduling
Industry 4.0
Journal
IF:
10
Papers:
7.9K
Citations:
3.6W
Organization
Cited Papers