.png){kind=icon}
返回
GARLSched: Generative adversarial deep reinforcement learning task scheduling optimization for large-scale high performance computing systems
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2022.04.032.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Efficient task scheduling has become increasingly complex as the number and type of tasks proliferate and the size of computing resource grows in large-scale distributed high-performance computing (HPC) systems. At present, deep reinforcement learning (DRL) methods have achieved certain success in scheduling problems. However, due to the exogeneity of the task and the sparsity of the reward, the learning of the DRL control policy requires a significant amount of training time and data and cannot guarantee effective convergence. Meanwhile, based on the understanding of HPC system characteristics, various scheduling policies with acceptable performance for different optimization goals have been developed by the experts. But these heuristic methods cannot adapt to environmental changes and optimize for specific workloads. Therefore, the generative adversarial reinforcement learning scheduling (GARLSched) algorithm is proposed to effectively guide the learning of DRL in large-scale dynamic scheduling issues based on the optimal policy in the expert pool. In addition, the task embedding-based discriminator network effectively improves and stabilizes the learning process. Experiments show that compared with heuristic and DRL scheduling algorithms, GARLSched can learn high-quality scheduling policies for various workloads and optimization objects. Furthermore, the learned models can perform stably even when applied to invisible workloads, making them more practical in HPC systems. (C) 2022 Elsevier B.V. All rights reserved.
Keyword:
Task scheduling
Deep reinforcement learning
Distributed systems
High performance computing
Expert guidance
期刊
F
IF:
6.1
论文数:
6.9K
被引数:
2.3W
机构
引用论文
.png){kind=icon}
Energy-Aware VM Consolidation in Cloud Data Centers Using Utilization Prediction Model基于利用率预测模型的云数据中心能耗感知虚拟机整合
PREAI
分享收藏Backfilling using system-generated predictions rather than user runtime estimates使用系统生成的预测而不是用户运行时估计进行回填
PREAI
分享收藏HUNTER: AI based holistic resource management for sustainable cloud computingHUNTER: 基于AI的整体资源管理，实现可持续的云计算
OAAI
OAAI分享收藏
查看更多内容