论文标题
数据中心网络负载平衡的多代理增强学习
Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center
论文作者
论文摘要
本文提出了网络负载平衡问题，这是多项式增强学习（MARL）方法的一项具有挑战性的现实任务。传统的启发式解决方案（例如加权成本多路径（WCMP）和当地最短队列（LSQ））对不断变化的工作负载分布和到达率的灵活性较小，并且在多个负载平衡器之间的平衡差。合作网络负载平衡任务被提出为DEC-POMDP问题，该问题自然会诱导MARL方法。为了弥合现实差距用于应用基于学习的方法，所有方法均已直接训练和评估来自中度至大规模的仿真系统。对现实测试床的实验表明，独立和“自私”负载平衡策略不一定是全球最佳的，而所提出的MARL解决方案在不同的现实设置上具有出色的性能。此外，分析了MAL方法在网络负载平衡中的潜在困难，这有助于吸引学习和网络社区的注意力。
This paper presents the network load balancing problem, a challenging real-world task for multi-agent reinforcement learning (MARL) methods. Traditional heuristic solutions like Weighted-Cost Multi-Path (WCMP) and Local Shortest Queue (LSQ) are less flexible to the changing workload distributions and arrival rates, with a poor balance among multiple load balancers. The cooperative network load balancing task is formulated as a Dec-POMDP problem, which naturally induces the MARL methods. To bridge the reality gap for applying learning-based methods, all methods are directly trained and evaluated on an emulation system from moderate-to large-scale. Experiments on realistic testbeds show that the independent and "selfish" load balancing strategies are not necessarily the globally optimal ones, while the proposed MARL solution has a superior performance over different realistic settings. Additionally, the potential difficulties of MARL methods for network load balancing are analysed, which helps to draw the attention of the learning and network communities to such challenges.
