Solving quay wall allocation problems based on deep reinforcement learning

Young-in Cho

doi:10.1016/j.engappai.2025.110598

Return

Solving quay wall allocation problems based on deep reinforcement learning

2025-06-01

PRE

DOI：10.1016/j.engappai.2025.110598

Original

Original request for help

Save

Abstract

En 中文

Quay walls and graving docks are critical production resources in shipyards. Traditionally, quay walls have not been a bottleneck resource for constructing conventional vessels, such as oil carriers and container ships. However, the growing demand for high value-added vessels requiring more complex post-stage outfitting operations has increased workloads at quay walls. Accordingly, the importance of efficient quay wall allocation has grown significantly to improve overall production efficiency and ensure timely vessel delivery. In this study, the quay wall allocation problem is modeled as a flexible job shop scheduling problem, incorporating machine preferences and preemption conditions. Notably, the uncertainty in vessel launching dates caused by delays in the erection process at graving docks is considered in the scheduling problems. To address the dynamic quay wall allocation problems, this study develops a dynamic quay wall allocation algorithm based on deep reinforcement learning, which adaptively allocates vessels to quay walls based on the working status of quay walls and the progress of outfitting operations. For this purpose, a novel Markov decision process is proposed, where a compound state representation composed of heterogeneous graphs and auxiliary matrices is devised to capture the complex relationships between quay walls and outfitting operations. In addition, an extended scheduling action space incorporating operation interruptions is defined, which can effectively utilize preemption conditions to enhance the scheduling performance. The performance of the proposed algorithm is evaluated through extensive numerical experiments based on test instances generated from real-world shipyard data under various environmental conditions. Experimental results demonstrate that the proposed algorithm consistently outperforms traditional rule-based heuristics and exhibits superior scalability compared to genetic programming, making it a promising solution for large-scale quay wall allocation problems.

Keywords:

Deep reinforcement learning

Heterogeneous graph

Flexible job-shop scheduling problem

Post-stage outfitting process

Shipbuilding