Impact of Forecast-Aware Observations and Attention Mechanisms in a Multi-Agent Deep Reinforcement Strategy for Voltage Control
Alan Lizarazo, G. Osma-Pinto, César Duarte Escuela de Ingenierías Eléctrica, Electrónica y de Telecomunicaciones (E3T), Universidad Industrial de Santander (UIS), Bucaramanga, Colombia.   2026-06-27

Abstract

High DER penetration, particularly PV, induces sub-hourly voltage excursions and reverse power flows that are difficult to mitigate with slow and discrete devices (OLTCs and switched capacitor banks). We study a multi-agent soft actor–critic (MASAC) voltage controller implemented in PyTorch and quantify the impact of two components: (i) forecast-aware local observations (Fc) using an oracle 1-hour-ahead PV forecast summary and (ii) an attention-based centralized critic (Att) that exploits the electrical neighborhood structure. The environment is simulated with AC power flow in pandapower using static ZIP loads and device capability constraints. Daily trajectories are built as 24-h load/PV pairs 288 at Δ��=5 min; training uses four trajectories. Four variants (NoFc/NoAtt, Fc/NoAtt, NoFc/Att, Fc/Att) are compared in a 2×2 design on a modified IEEE 33-bus (12.66 kV) feeder with one OLTC, three capacitor banks, and six PV inverters. Inverters update every step, while OLTC/CB actions are gated hourly. Scenario A evaluates the same four pairs, while Scenario B tests generalization on 24 unseen pairs. In Scenario B, Fc/Att attains the lowest average voltage deviation (0.0161 p.u.), the smallest out-of-band fraction (6.5%), and the lowest daily losses (1.145 MWh), resulting in the highest episode return under the proposed reward.

Key words: Voltage control, active distribution networks, multi-agent reinforcement learning, attention, forecasting.

References

[1]           P. Srivastava, R. Haider, V. Nair, V. Venkataramanan, A. M. Annaswamy, and A. K. Srivastava, “Voltage regulation in distribution grids: A survey,” Annual Reviews in Control, vol. 55, pp. 165–181, 2023, doi: 10.1016/j.arcontrol.2023.03.008.

[2]           A. Savasci, A. Inaolaji, and S. Paudyal, “Optimal tuning of local voltage control rule of load tap changers for dynamic operation of unbalanced distribution networks,” IEEE Transactions on Industry Applications, vol. 60, no. 1, pp. 1322–1331, 2024, doi: 10.1109/TIA.2023.3313128.

[3]           Y. Liu, J. Bebic, B. Kroposki, J. de Bedout, and W. Ren, “Distribution system voltage performance analysis for high-penetration PV,” in 2008 IEEE energy 2030 conference, 2008, pp. 1–8. doi: 10.1109/ENERGY.2008.4781069.

[4]           Y. Gui et al., “Voltage support with PV inverters in low-voltage distribution networks: An overview,” IEEE journal of emerging and selected topics in power electronics, vol. 12, no. 2, pp. 1503–1522, 2024, doi: 10.1109/JESTPE.2023.3280926.

[5]           C. W. G. C6.31, “Smart inverters for distribution systems,” CIGRE Technical Brochure, vol. No. 837, 2020.

[6]           F. Matsushima, M. Aoki, Y. Nakamura, S. C. Verma, K. Ueda, and Y. Imanishi, “Multi-timescale voltage control method using limited measurable information with explainable deep reinforcement learning,” Energies, vol. 18, no. 3, p. 653, 2025, doi: 10.3390/en18030653.

[7]           H. Wang, Y. Yao, J. Zhao, and F. Ding, “Data-driven mean-corrected recursive estimation-based optimal DER dispatch for distribution system voltage control,” IEEE Transactions on Sustainable Energy, vol. 16, no. 4, pp. 2861–2873, 2025, doi: 10.1109/TSTE.2025.3570218.

[8]           X. Zhang, Z. Wu, Q. Sun, W. Gu, S. Zheng, and J. Zhao, “Application and progress of artificial intelligence technology in the field of distribution network voltage control: A review,” Renewable & Sustainable Energy Reviews, vol. 192, p. 114282, 2024.

[9]           Y. Zheng, H. Zhang, Z. Long, S. Gao, Q. Yang, and H. Ji, “Data-driven multi-mode adaptive control for distribution networks with multi-region coordination,” Processes, vol. 13, no. 10, 2025, doi: 10.3390/pr13103046.

[10]         N. R. Chowdhury, A. K. Rao, Y. Bichpuriya, V. Sarangan, and S. A. Soman, “Provably robust online voltage control for distribution networks with line parameter estimation,” European Journal of Control, vol. 85, p. 101252, 2025, doi: 10.1016/j.ejcon.2025.101252.

[11]         W. Jiao et al., “Analytical target cascading based real-time distributed voltage control for MV and LV active distribution networks,” International Journal of Electrical Power & Energy Systems, vol. 159, p. 110024, 2024, doi: 10.1016/j.ijepes.2024.110024.

[12]         Z. Zhang, Z. Y. Dong, and D. Yue, “Multiple time-scale voltage regulation for active distribution networks via three-level coordinated control,” IEEE Transactions on Industrial Informatics, vol. 20, no. 3, pp. 4429–4439, 2024, doi: 10.1109/TII.2023.3304340.

[13]         X. Sun and J. Qiu, “Two-stage volt/VAR control in active distribution networks with multi-agent deep reinforcement learning method,” IEEE Transactions on Smart Grid, vol. 12, no. 4, pp. 2903–2912, 2021, doi: 10.1109/TSG.2021.3052998.

[14]         D. Cao et al., “Deep reinforcement learning enabled physical-model-free two-timescale voltage control method for active distribution systems,” IEEE Transactions on Smart Grid, vol. 13, no. 1, pp. 149–165, 2022, doi: 10.1109/TSG.2021.3121792.

[15]         H. Peng et al., “Deep reinforcement learning based multi-timescale volt/VAR control in distribution networks considering network reconfiguration,” IEEE Transactions on Sustainable Energy, vol. 16, no. 4, pp. 2948–2958, 2025, doi: 10.1109/TSTE.2025.3574806.

[16]         S. Toledo-Cortés, J. S. Lara, A. Zambrano, F. A. G. Osorio, and J. R. Garcia, “Characterization of electricity demand based on energy consumption data from colombia,” International Journal of Electrical and Computer Engineering (IJECE), vol. 13, no. 5, pp. 4798–4809, 2023, doi: 10.11591/ijece.v13i5.pp4798-4809.