Centralized and Collaborative RL-Based Resource Allocation in Virtualized Dynamic Fog Computing

Fog computing (FC) emerged as a new paradigm enabling the deployment of new Internet of Things (IoT) applications. Fog infrastructure is composed of heterogeneous nodes characterized by a complex distribution, mobility, and sporadic resource availability. Hence, resource coordination for continuous Quality-of-Service (QoS) satisfaction becomes challenging, and accurate resource tracking is needed for flawless servicing. In this context, we investigate and propose online resource allocation solutions. The main objective is to maximize the number of satisfied users within a predefined latency requirement. Hence, we model the FC environment as a Markov Decision Process, and then, we formulate the optimization problem. Due to the problem's NP-hardness, we leverage the reinforcement learning (RL) tool to develop resource allocation schemes. First, a centralized method where a smart fog controller possesses a global awareness of the FC environment is proposed. Next, a more practical and collaborative solution is presented, where each RL-enabled agent manages a group of fog nodes and their resources in order to satisfy computing requests. Based on real-world mobility data sets, simulation results illustrate the high efficiency of the proposed solutions with a preference for the collaborative approach. The superiority of our proposed solutions over state-of-the-art methods is also illustrated.

Automated summary

Fog computing is a new paradigm for enabling IoT applications. Due to the complexity of fog infrastructure, resource coordination and tracking are challenging. In this study, we propose online resource allocation solutions to maximize user satisfaction within latency requirements, using Markov Decision Process and reinforcement learning. Simulation results based on real-world data demonstrate the high efficiency of our collaborative solution compared to existing methods.