Smart Charging of Urban Electric Bus Fleets Using Proximal Policy Optimization
A. Martínez(1), M. Alonso(1), H. Amaris(1), M. Cordeiro-Costas(2) 1. Department of Electrical Engineering, Universidad Carlos III de Madrid, Leganés (Spain). 2. CINTECX, Universidade de Vigo, 36310, Vigo (Spain) 2026-06-27

Abstract

The large-scale deployment of electric buses (e-buses) requires intelligent charging hubs capable of ensuring cost-effective operation while maintaining fleet availability and preventing grid congestion. This paper presents a reinforcement learning–based energy management system for an urban e-bus logistics depot centre operating under technical and economic constraints, including uncertainty in battery state of charge and operational schedules. The charging problem is formulated as a Markov Decision Process and solved using Proximal Policy Optimization (PPO). The agent determines charger allocation and hourly energy delivery based on bus-level variables (state of charge, remaining energy, time to departure, priority) and global variables (time, accumulated depot consumption, dynamic electricity prices, and charger availability). The reward function balances timely charging, grid compliance, priority service, and cost minimization. Validation on a realistic depot with 16 chargers and 25 e-buses demonstrates that the proposed controller satisfies fleet requirements while respecting network limits and shifting demand away from high-price periods.

Key words: Smart charging, reinforcement learning, e- bus fleet charging, grid-constraint optimization, PPO.

References

[1] J. Guanetti, Y. Kim, X. Shen, J. Donham, S. Alexander, B. Wootton and F. Borrelli, “Increasing Electric Vehicles Utilization in Transit Fleets using Learning, Predictions, Optimization, and Automation,” arXiv preprint arXiv:2305.14732, 2023.

[2] J. Fan, H. Wang and A. Liebman, “MARL for Decentralized Electric Vehicle Charging Coordination with V2V Energy Exchange,” arXiv preprint arXiv:2308.14111, 2023.

[3] J. Qi, L. Lei, T. Jonsson and D. Niyato, “Optimizing Electric Bus Charging Scheduling with Uncertainties Using Hierarchical Deep Reinforcement Learning,” arXiv preprint, 2025.

[4] P. Michailidis, T. Stergiou and G. Ingramidis, “Reinforcement Learning for Electric Vehicle Charging Systems: Fleet-Centric Approaches,” Energies, vol. 18, 2025.

[5] J. A. Manzolli, J. P. F. Trovão and C. H. Antunes, “Electric Bus Fleet Charging Management: A Robust Optimisation Framework Addressing Battery Ageing and Uncertainty,” Energy, 2025.

[6] Red Eléctrica de España. Markets and prices. e·sios [online] (2026).