arrow
Return

Learning to Optimize Workflow Scheduling for an Edge-Cloud Computing Environment

delete2024-07-01
delete1
PRE
AI
K
Kaige Zhu
张振江 (Zhenjiang Zhang) *
S
Sherali Zeadally
孙锋 cover
孙锋 (Feng Sun)
DOI:10.1109/TCC.2024.3408006delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
The widespread deployment of intelligent Internet of Things (IoT) devices brings tighter latency demands on complex workload patterns such as workflows. In such applications, tremendous dataflows are generated and processed in accordance with specific service chains. Edge computing has proven its feasibility in reducing the traffic in the core network and relieving cloud datacenters of fragmented computational demands. However, the efficient scheduling of workflows in hybrid edge-cloud networks is still challenging for the intelligent IoT paradigm. Existing works make dispatching decisions prior to real execution, making it difficult to cope with the dynamicity of the environment. Consequently, the schedulers are affected both by the scheduling strategy and by the mutual impact of dynamic workloads. We design an intelligent workflow scheduler for use in an edge-cloud network where workloads are generated with continuous steady arrivals. We develop new graph neural network (GNN)-based representations for task embedding and we design a proximal policy optimization (PPO)-based online learning scheduler. We further introduce an intrinsic reward to obtain an instantaneous evaluation of the dispatching decision and correct the scheduling policy on-the-fly. Numerical results validate the feasibility of our proposal as it outperforms existing works with an improved quality of service (QoS) level.
Keywords:
Task analysis
Processor scheduling
Dynamic scheduling
Cloud computing
Internet of Things
Edge computing
Job shop scheduling
Workflow scheduling
edge computing
reinforcement learning

Journal

I
IEEE Transactions on Cloud Computing
IF:
5
Papers:
1.8K
Citations:
4.3K

Organization

B
Beijing Jiaotong University
Scholars:
2.2W
Papers: 1.7W
Citations: 1.2W
U
University of Kentucky
Scholars:
2.5W
Papers: 2.1W
Citations: 41