An Online Learning-Based Intelligent Dynamic Resource Allocation for Reconfigurable Intelligent Surface Assisted Wireless Network
Loading...
Authors
Zhang, Yuzhu
Issue Date
2024
Type
Dissertation
Language
Keywords
Alternative Title
Abstract
In today's era of information globalization, the demand for wireless network data capacity is rapidly increasing. The available frequency resources of millimeter waves (mmWave) far exceed those in existing communication systems, alleviating the spectrum shortage issue and thus attracting widespread research attention. However, high-frequency communication also has its drawbacks, namely significant signal path attenuation, posing challenges to its practical application. Reconfigurable Intelligent Surfaces (RIS) is an emerging technology that can address path attenuation issues in mmWave communication and enhance communication performance, thus becoming a key candidate technology for future 6 th Generation (6G) mobile communication systems. However, RIS-assisted wireless communication also faces many new challenges, especially in designing low-complexity beamforming schemes to jointly optimize the passive beams of the RIS and the active beams of the Base Stations (BSs), as the RIS system may require a large energy supply, especially when frequent adjustments of the phase and amplitude of the reflecting elements are needed, which may lead to increased energy consumption. This dissertation conducts in-depth research around the challenges mentioned above and has achieved several innovative research outcomes, briefly summarized as follows: First, the optimal resource allocation problem in reconfigurable intelligent surface (RIS) assisted dynamic wireless networks with uncertain time-varying wireless channels was investigated. We represented the RIS-assisted wireless communication network with dynamic wireless channels as a state-space model. Then, the optimal resource allocation problem was formulated as a finite-time joint optimal control of users' transmit powers and RIS phase shifts. Next, due to the time-varying and uncertain nature of wireless channels, a novel online reinforcement learning technique, Actor-Critic design, was developed along with neural networks (NN) to learn the optimal resource allocation policies in real-time. Because capacity is limited due to the RIS's hardware constraints, we have developed and utilized a novel online data-enabled Actor-Critic-Obstacle reinforcement learning algorithm combined with neural networks (NNs) to learn optimal transmit power control and RIS phase control strategies under hardware limitations. Next, we extended the utilization of RIS to additional scenarios. In the cases of multiple users and multiple RISs, we explored the decentralized dynamic resource allocation optimization problem in self-organizing network communication supported by RISs and proposed cooperation learning based on the Twin Delayed Deep Deterministic Policy Gradient algorithm (TD3). Additionally, we addressed the communication resource allocation problem of the social Internet of Things (SIoT) in optimizing electric vehicle charging networks using causal-factor-based reinforcement learning.