A job scheduling algorithm based on parallel workload prediction on computational grid

Xiaoyong Tang

doi:10.1016/j.jpdc.2022.09.007

2023-01-01

PRE

DOI：10.1016/j.jpdc.2022.09.007

原文链接

原文求助

摘要

En 中文

Generally, the computational grid consists of a large number of computing nodes, some of them are idle due to the uneven geographical distribution of computing requirements. This may cause workload unbalancing problems, which affect the performance of large-scale computational grids. In order to balance the computing requirements and computing nodes, we propose a job scheduling algorithm based on the workload prediction of computing nodes. We first analyze the causes of workload imbalance and the feasibility of reallocating computing resources. Secondly, we design an application and workload -aware scheduling algorithm (AWAS) by combining the previously designed workload prediction model. To reduce the complexity of the AWAS algorithm, we propose a parallel job scheduling method based on computing node workload prediction. The experiments show that the AWAS algorithm can balance the workload among different computing nodes on the real-world dataset. In addition, we propose the parallelism of workload prediction model from the perspective of internal structure and data set to make AWAS apply to more computing nodes of the large-scale computing grids. Experimental results show that the combination of the two can achieve satisfactory acceleration efficiency.(c) 2022 Elsevier Inc. All rights reserved.

Keyword:

Job scheduling

Workload balancing

Neural network

Computational grid

Parallel

期刊

Journal of Parallel and Distributed Computing

IF:

论文数:

3.8K

被引数:

4.8K

机构

Singapore Institute of Technology

学者数:

848

论文数: 772

被引数: 817

shanghai university

学者数:

3.9W

论文数: 2.7W

被引数: 52

引用论文

Task scheduling and resource allocation in cloud computing using a heuristic approach

JOURNAL OF CLOUD COMPUTING-ADVANCES SYSTEMS AND APPLICATIONS

IF4.3

2018-02-08

123

OAAI

Gawali, Mahendra Bhatu; Shinde, Subhash K.

Multi-target deep neural networks: Theoretical analysis and implementation

NEUROCOMPUTING

IF6.5

2018-01-01

OAAI

Zeng, Zeng; Liang, Nanying; Yang, Xulei; Hoi, Steven

A Survey of Low-Energy Parallel Scheduling Algorithms

ieee transactions on cloud computing

IF3.9

2022-01-01

PREAI

Xie, Guoqi; Xiao, Xiongren; Peng, Hao; Li, Renfa; Li, Keqin

Supercomputer-Based Ensemble Docking Drug Discovery Pipeline with Application to Covid-19

JOURNAL OF CHEMICAL INFORMATION AND MODELING

IF5.3

2020-12-16

130

OAAI

Acharya, A.; Agarwal, R.; Baker, M. B.; Baudry, J.; Bhowmik, D.; Boehm, S.; Byler, K. G.; Chen, S. Y.; Coates, L.; Cooper, C. J.; Demerdash, O.; Daidone, I; Eblen, J. D.; Ellingson, S.; Forli, S.; Glaser, J.; Gumbart, J. C.; Gunnels, J.; Hernandez, O.; Irle, S.; Kneller, D. W.; Kovalevsky, A.; Larkin, J.; Lawrence, T. J.; LeGrand, S.; Liu, S-H; Mitchell, J. C.; Park, G.; Parks, J. M.; Pavlova, A.; Petridis, L.; Poole, D.; Pouchard, L.; Ramanathan, A.; Rogers, D. M.; Santos-Martins, D.; Scheinberg, A.; Sedova, A.; Shen, Y.; Smith, J. C.; Smith, M. D.; Soto, C.; Tsaris, A.; Thavappiragasam, M.; Tillack, A. F.; Vermaas, J. V.; Vuong, V. Q.; Yin, J.; Yoo, S.; Zahran, M.; Zanetti-Polzi, L.

Multinational Corporations in a Nationalistic World

Challenge

IF0

2015-10-09

PREAI

Howe Martyn

Acute Basilar Artery Dissection Treated by Emergency Stenting in a 13-Year-Old Boy

Pediatric Neurosurgery

IF0

2005-11-18

PREAI

Masaki Komiyama; Masaki Yoshimura; Yuji Honnda; Yasuhiro Matsusaka; Toshihiro Yasui