A Batch-Constrained Safe Deep Q-Learning-Based Cloud-Edge Collaborative Framework for Dynamic Operation Optimization in Industrial Processes

Xin Qin

doi:10.1109/TII.2025.3609186

Return

A Batch-Constrained Safe Deep Q-Learning-Based Cloud-Edge Collaborative Framework for Dynamic Operation Optimization in Industrial Processes

2025-10-01

PRE

DOI：10.1109/TII.2025.3609186

Original

Original request for help

Save

Abstract

En 中文

Operation optimization of industrial processes is essential for high-quality industrial development, which involves production, quality, cost, safety, and sustainability. However, the complexity, dynamic uncertainty, inconsistent data quality, and limited computational resources of industrial processes pose significant challenges for operation optimization. To address these issues, we propose a batch-constrained safe deep Q-learning-based cloud-edge collaborative framework for dynamic operation optimization. First, a performance-traceable data space is constructed based on spatio-temporal alignment of equipment data, which is embedded with format validation, boxplot–based outlier detection, and mean-value filling algorithms to enhance data quality and support high-level applications. Second, a batch-constrained safe deep Q-learning algorithm is developed, in which the agent uses a conditional variational autoencoder to generate candidate actions matching the historical data distribution, and applies a safety protection strategy, perturbation model, and double Q-networks to explore safe optimization policies, thereby eliminating extrapolation error in offline reinforcement learning and ensuring the safety of operation optimization. Then, a novel parallel informer and long-short term memory model is constructed to capture process features across multiple time scales for online performance prediction. Whenever process performance deviates from the specifications, the actor network is immediately triggered to generate a dynamic optimization policy, which is then fed back into the performance prediction model for evaluation, ensuring reliable optimization outcomes. Finally, by leveraging a cloud–edge architecture to address computational resource allocation and efficiency, we apply the proposed framework to a real hot strip mill process, demonstrating its effectiveness and safety.

Keywords:

Batch-constrained safe deep Q-learning

cloud–edge collaboration

dynamic operation optimization

industrial process

reinforcement learning (RL)