eNB Selection for Machine Type Communications Using Reinforcement Learning Based Markov Decision Process

doi:10.1109/TVT.2017.2730230

2017-12-01

PRE

Yu-Jui Liu

Shin‐Ming Cheng *

Yu-Lin Hsueh

DOI：10.1109/TVT.2017.2730230

原文链接

原文求助

摘要

En 中文

Machine type communication (MTC), as one of the most promising technologies in the future wireless communication, has brought mobile communication network into a new level. The breakthrough of cutting-edge technology and broad coverage of cellular networks in long term evolution advanced network constitute an ideal platform for ubiquitous MTC service provisioning on a large scale. However, under the traditional attach approach, the massive MTC devices always select the evolved NodeB (eNB) with the best signal quality for the attachment, thereby causing the network congestion and overload. As a result, it is necessary to design an efficient eNB selection scheme to avoid overload issue. In this paper, by modeling MTC arrivals using nonhomogeneous poisson process with the memoryless property, we formulate the eNB selection problem as a Markov decision process (MDP) and try to compute an optimal solution. Since the network parameters for MDP are not obtained easily, a learning-based algorithm with value-difference based exploration (VDBE) policy who could leverage both present conditions and expected future demands is further proposed. The performances of the proposed reinforcement learning (RL) VDBE, other RL-based, the legacy best-signal-quality, and MDP without RL eNB-selection schemes are analyzed in terms of blocking probability, transmission rate, and load balancing. The simulation results show that our scheme has the best performance on the blocking probability and loading balancing while an acceptable result on the transmission rate. It is suitable for the MTC environment with a highly changed number of MTC arrivals while without high requirement on the transmission rate.

Keyword:

eNB selection

machine type communications

Markov decision process

reinforcement learning

value-difference based exploration