Joint Trajectory Design and Phase Shift Optimization for Multi-RIS-Assisted UAV Relay Network Using Deep Reinforcement Learning

Wei Chen

doi:10.1109/JIOT.2024.3509514

Joint Trajectory Design and Phase Shift Optimization for Multi-RIS-Assisted UAV Relay Network Using Deep Reinforcement Learning

2024-01-01

0

PRE

AI

DOI：10.1109/JIOT.2024.3509514

Original

Original request for help

Save

Abstract

En 中文

This article investigates an uncrewed aerial vehicle (UAV) relay network in the urban Internet of Things (IoT) environment designed to facilitate communications for ground terminals (GTs) by transmitting their data to a remote base station (BS) using the decode and forward (DF) protocol, where the orthogonal frequency division multiple access (OFDMA) is adopted for the GTs' transmissions. To address the challenges posed by building obstructions on radio propagation, we incorporate the multiple reconfigurable intelligent surfaces (RISs) to improve the quality of the GTs-UAV and UAV-BS links. The primary objective is to maximize the sum rate of all the GTs through the joint optimization of UAV trajectory, GTs transmit power, subchannel allocation, and multi-RIS phase shifts. To this end, we reformulate the optimization problem as a Markov decision process (MDP) and propose a deep reinforcement learning (DRL) approach for addressing our formulated problem, called the soft actor critic-based subchannel allocation and phase shift optimization with trajectory design (SAC-CAPSTD) algorithm. By continuously interacting with the environment, the proposed system refines its policy to determine the optimal UAV flight trajectory and the subchannel allocation strategy for GTs within their transmit power constraints. Concurrently, a discrete phase shift optimization method is implemented to adjust the phase shift for each RIS element. Finally, numerical results confirm that the proposed SAC-CATDPS algorithm can significantly achieve higher sum rate of all the served GTs and exhibit better convergence performance compared with the benchmark DRL-based algorithms.

Keywords:

Autonomous aerial vehicles

Trajectory

Optimization

Relay networks

Resource management

Reconfigurable intelligent surfaces

Internet of Things

Array signal processing

Wireless communication

Heuristic algorithms

Deep reinforcement learning (DRL)

reconfigurable intelligent surface (RIS)

relay networks

uncrewed aerial vehicle (UAV)

Joint Trajectory Design and Phase Shift Optimization for Multi-RIS-Assisted UAV Relay Network Using Deep Reinforcement Learning

Abstract

Journal

Organization