Journal
SIGCOMM '21: PROCEEDINGS OF THE 2021 ACM SIGCOMM 2021 CONFERENCE
Volume -, Issue -, Pages 258-271Publisher
ASSOC COMPUTING MACHINERY
DOI: 10.1145/3452296.3472902
Keywords
Network planning; Reinforcement learning; Graph neural network
Categories
Funding
- NSF [CNS-1813487, CCF-1918757]
- PKU-Baidu Fund [2020BD007]
Ask authors/readers for more resources
This paper proposes a deep reinforcement learning approach called NeuroPlan to solve the network planning problem, which involves multi-step decision making and cost minimization. By utilizing graph neural network and domain-specific techniques, the method can scale to large topologies and reduce costs by up to 17% compared to hand-tuned heuristics.
Network planning is critical to the performance, reliability and cost of web services. This problem is typically formulated as an Integer Linear Programming (ILP) problem. Today's practice relies on hand-tuned heuristics from human experts to address the scalability challenge of ILP solvers. In this paper, we propose NeuroPlan, a deep reinforcement learning (RL) approach to solve the network planning problem. This problem involves multi-step decision making and cost minimization, which can be naturally cast as a deep RL problem. We develop two important domain-specific techniques. First, we use a graph neural network (GNN) and a novel domain-specific node-link transformation for state encoding, in order to handle the dynamic nature of the evolving network topology during planning decision making. Second, we leverage a two-stage hybrid approach that first uses deep RL to prune the search space and then uses an ILP solver to find the optimal solution. This approach resembles today's practice, but avoids human experts with an RL agent in the first stage. Evaluation on real topologies and setups from large production networks demonstrates that NeuroPlan scales to large topologies beyond the capability of ILP solvers, and reduces the cost by up to 17% compared to hand-tuned heuristics.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available