Leakage-Safe Evaluation for Day-Ahead Load Forecasting Under an Operational Data Gap Constraint based on a Minimal Information Set
Luis Carreño-Barrera, Jairo Blanco-Solano, Cesar Duarte School of Electrical, Electronic and Telecommunications Engineering (E3T). Universidad Industrial de Santander (UIS),Bucaramanga, Colombia 2026-06-27

Abstract

End-to-end pipelines with data leakage can mislead model selection in day-ahead load forecasting. When training and evaluation inadvertently violate the information set available at issuance time, performance estimates cease to be decision-relevant. We quantify this effect by contrasting a leakage-safe (deployment-like) protocol against common leaky evaluation pitfalls for three base learners: Ridge regression, Light Gradient Boosting Machine (LightGBM), and a Long Short-Term Memory (LSTM) network. We adopt a forecast-free information set that enforces an operational delay (gap) of g = 18 h at issuance time t0; i.e., models only use observations available up to t0 – g together with deterministic calendar descriptors (hour-of-day, day-of-week, holidays) known at issuance time. Importantly, we intentionally restrict inputs to this minimal, deployment-robust feature set (load history + calendar only, no exogenous variables or forecasts), yet still obtain competitive accuracy. Load forecasts are evaluated through a deployment-like day-ahead simulator with a fixed daily issuance time, and learning-based models are trained in residual form relative to a Naive-week (weekly persistence) baseline model. Across three Colombian served-load datasets—the national interconnected bulk power system and two distribution system operator (DSO) electricity markets—split contamination can change conclusions and inflate accuracy. Under the proposed leakage-safe protocol on the medium-sized DSO market, Ridge yields the lowest mean absolute percentage error (MAPE), 2.13%, among the three learners, while leaky split contamination can make LightGBM appear unrealistically strong (MAPE = 0.64%). In contrast, Ridge is comparatively leakage-robust, showing only marginal changes between leakage-safe and contaminated evaluations, unlike higher-capacity learners. Despite using only load-and-calendar inputs, Ridge also matches or surpasses the DSO market forecast across the studied areas. We additionally compute mean absolute error (MAE) and root mean squared error (RMSE) to complement MAPE.

Key words: Day-ahead load forecasting; leakage-safeevaluation; forecast-free information set; operational gap; data leakage.

References

[1] T. Hong and S. Fan, “Probabilistic electric load forecasting: A tutorial review,” International Journal of Forecasting, vol. 32, no. 3, pp. 914–938, 2016, 10.1016/j.ijforecast.2015.11.011

[2] S. Fan and R. J. Hyndman, “Short-term load forecasting based on a semi-parametric additive model,” IEEE Transactions on Power Systems, vol. 27, no. 1, pp. 134–141, 2012, 10.1109/TPWRS.2011.2162082

[3] S. Ben Taieb and R. J. Hyndman, “A gradient boosting approach to the kaggle load forecasting competition,” International Journal of Forecasting, vol. 30, no. 2, pp. 382–394, 2014, 10.1016/j.ijforecast.2013.07.005

https://gestornormativo.creg.gov.co/gestor/entorno/docs/resolucion_creg_0025_1995.htm

[5] Comisión de Regulación de Energía y Gas (CREG), “Resolución CREG 100 de 2019 [CREG resolution 100 of 2019],” 2019, [Online]. Available: https://gestornormativo.creg.gov.co/gestor/entorno/docs/resolucion_creg_0100_2019.htm

[6] Unidad de Planeación Minero Energética (UPME), “Generation Projects Registry – July 2024,” Jul. 2024, [Online]. Available: https://www1.upme.gov.co/siel/Inscripcion_proyectos_generacion/Registro_julio_2024.pdf

[7] XM Compañía de Expertos en Mercados S.A. E.S.P., “En 2024 se declararon en operación 67 proyectos de generación y 32 de transmisión [In 2024, 67 generation projects and 32 transmission projects entered operation],” 2025, [Online]. Available: https://www.valoraanalitik.com/cuantos-proyectos-de-energia-en-colombia-entraron-en-2024/

[8] L. Tashman, “Out-of-sample tests of forecasting accuracy: An analysis and review,” International Journal of Forecasting, vol. 16, no. 4, pp. 437–450, 2000, 10.1016/S0169-2070(00)00065-0

[9] C. Bergmeir, R. J. Hyndman, and B. Koo, “A note on the validity of cross-validation for evaluating autoregressive time series prediction,” Computational Statistics & Data Analysis, vol. 120, pp. 70–83, 2018, 10.1016/j.csda.2017.11.003

[10] M. López de Prado, Advances in Financial Machine Learning. Hoboken, NJ: Wiley, 2018.

[11] J. W. Taylor and P. E. McSharry, “Short-term load forecasting methods: An evaluation based on European data,” IEEE Transactions on Power Systems, vol. 22, no. 4, pp. 2213–2219, 2007, 10.1109/TPWRS.2007.907583

[12] Q. Zhao, X. Liu, and J. Fang, “Extreme gradient boosting model for day-ahead STLF in national level power system: Estonia case study,” Energies, vol. 16, no. 24, p. 7962, 2023, 10.3390/en16247962

[13] P. Netzell, H. Kazmi, and K. Kyprianidis, “Deriving input variables through applied machine learning for short-term electric load forecasting in Eskilstuna, Sweden,” Energies, vol. 17, no. 10, p. 2246, 2024, 10.3390/en17102246

[14] S. Kaufman, S. Rosset, and C. Perlich, “Leakage in data mining: Formulation, detection, and avoidance,” ACM Transactions on Knowledge Discovery from Data, vol. 6, no. 4, pp. 1–21, 2012, 10.1145/2382577.2382579

[15] P. Subotić, U. Bojanić, and M. Stojić, “Statically detecting data leakages in data science code,” in Proceedings of the 11th ACM SIGPLAN International Workshop on the State Of the Art in Program Analysis (SOAP ’22). Association for Computing Machinery, 2022.

[16] S. Albelali and M. Ahmed, “Hidden leaks in time series forecasting: How data leakage affects LSTM evaluation across configurations and validation strategies,” arXiv:2512.06932, 2025, arXiv:2512.06932

[17] T. Talagala, “tsdataleaks: An R package for detecting data leaks in forecasting competitions,” 2024, https://cran.r-project.org/package=tsdataleaks

[18] W. Feng, R. Tao, J. Cartlidge, and J. Zheng, “Vmdnet: Time series forecasting with leakage-free samplewise variational mode decomposition and multibranch decoding,” arXiv:2509.15394, 2025, arXiv:2509.15394

[19] M. Meyer, S. Kaltenpoth, K. Zalipski, H. Albers, and O. Müller, “Ts-arena: A pre-registered live forecasting platform,” 2025, arXiv:2512.20761