Machine Learning Algorithms for Resource Management of HetNets and Cellular-connected UAVs

Abstract

With the limited bandwidth and capacity available in cellular networks caused by the tremendous increase in connected devices and services, resource allocations such as power and spectrum are crucial. Heterogeneous Networks (HetNets) provide an efficient solution to leverage spectrum sharing and power allocation. The main objective of our research is to introduce Machine Learning (ML), specifically, Reinforcement Learning (RL), as a game-changer in wireless communication, and to evaluate the challenges of power allocation for cognitive and non-cognitive femtocells. Spectrum sharing offers an efficient solution by allowing secondary User Equipment (SU) access the resources made available by the Primary Base station (PBS). In our models, we consider downlink analysis of two-tier HetNet consisting of a single macrocell overlaid by multiple femtocells, several Primary UEs (PU) and indoor SU. The primary focus consists of two parts. First, we modeled unique power allocation schemes for two-tier femtocell HetNets and evaluated different RL algorithms. We derived two reward functions that guarantee fair capacity for PU and SU to meet their Quality of Service (QoS). Our analysis spans stochastic geometry rate coverage analysis of femtocell networks as a measure of network performance. Second, we introduced a novel paradigm that enables Beyond visual line of sight (BVLOS) x operations of unmanned aerial vehicles (UAVs) under stringent aviation regulations via Distributional RL algorithms. Moreover, we derived a non-convex optimization problem to minimize the overall round-trip latency between the UAV and the ground controller. The analysis and simulation results presented a comprehensive evaluation of the proposed schemes. Finally, we concluded this thesis with an overall evaluation and summary of our research, highlighting its importance, effectiveness, and possible improvement suggestions. In addition, we briefly discussed some future research directions.