Hybrid Deep Reinforcement Learning-Based Task Offloading for D2D-Assisted Cloud-Edge-Device Collaborative Networks

doi:10.1109/TMC.2024.3425723

2024-12-01

PRE

Wenhao Fan *

DOI：10.1109/TMC.2024.3425723

原文链接

原文求助

摘要

En 中文

In D2D (Device to Device)-assisted cloud-edge-device collaborative networks, the tasks of a busy device can be processed locally, offloaded to an idle device through D2D transmission, offloaded to the ES (Edge Server) of the affiliated BS (Base Station), further to the ES of another BS through ES-ES transmission, or further to the CS (Cloud Server) through ES-CS transmission. However, existing works did not fully consider both the D2D task offloading and cloud-edge-device collaboration. Moreover, the very high complexity of the joint resource optimization problem makes it extremely challenging to be solved efficiently by relying solely on numerical methods or machine-learning-based methods. In this paper, we propose a task offloading scheme to minimize the total system cost considering the time and energy consumption of all the devices. The task offloading decision, transmission power allocation, transmission rate allocation, and computational resource allocation are jointly optimized. We design a DRL (deep reinforcement learning)-based algorithm to solve the optimization problem efficiently through a hybrid approach, which decomposes the problem into sub-problems, and then jointly leverages an SD3 (Softmax Deep Double Deterministic Policy Gradients)-based DRL method to handle the task offloading sub-problem and uses multiple numerical methods to handle the other small-scale sub-problems. Extensive simulations are conducted in 7 scenarios. The superiority of our scheme is demonstrated in comparison with 4 reference schemes.

Keyword:

Task analysis

Resource management

Device-to-device communication

Collaboration

Cloud computing

Servers

Optimization

edge computing

task offloading

D2D

cloud computing