论文标题
基于分散逆变器的伏特控制的在线多代理增强学习
Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control
论文作者
论文摘要
已广泛研究了分布式伏特/VAR控制（VVC）方法的主动分布网络（ADN），该方法基于完美的模型和实时P2P通信。但是，该模型总是与重大参数错误不完整，并且很难维护此类P2P通信系统。在本文中，我们为VVC提出了一个在线多代理强化学习和分散的控制框架（OLDC）。在此框架中，VVC问题被提出为受约束的马尔可夫游戏，我们提出了一种新型的多代理限制的软演员 - 批评（MACSAC）增强算法。 MACSAC用于在线训练控制代理，因此不再需要准确的ADN模型。然后，训练有素的代理可以使用本地测量值实现分散的最佳控制，而无需实时P2P通信。带有MACSAC的OldC对各种计算和通信条件表现出非凡的灵活性，效率和鲁棒性。对IEEE测试用例的数值模拟不仅表明，所提出的MACSAC优于最先进的学习算法，而且还支持在线应用程序中OldC框架的优越性。
The distributed Volt/Var control (VVC) methods have been widely studied for active distribution networks(ADNs), which is based on perfect model and real-time P2P communication. However, the model is always incomplete with significant parameter errors and such P2P communication system is hard to maintain. In this paper, we propose an online multi-agent reinforcement learning and decentralized control framework (OLDC) for VVC. In this framework, the VVC problem is formulated as a constrained Markov game and we propose a novel multi-agent constrained soft actor-critic (MACSAC) reinforcement learning algorithm. MACSAC is used to train the control agents online, so the accurate ADN model is no longer needed. Then, the trained agents can realize decentralized optimal control using local measurements without real-time P2P communication. The OLDC with MACSAC has shown extraordinary flexibility, efficiency and robustness to various computing and communication conditions. Numerical simulations on IEEE test cases not only demonstrate that the proposed MACSAC outperforms the state-of-art learning algorithms, but also support the superiority of our OLDC framework in the online application.
