JCSIS
019928311823 info@jcsis.org
JCSIS

Deep Q-Network Reinforcement Learning for Real-Time Energy Storage Scheduling in Wind-Solar Hybrid Systems.

Najaad OubeBlika
Volume 5

Abstract

Real-time energy storage scheduling is a critical requirement for the reliable and economic operation of wind–solar hybrid energy systems, particularly because photovoltaic and wind power generation are inherently intermittent, weather-dependent, and difficult to predict with complete certainty. The complementary nature of solar and wind resources can improve renewable energy availability; however, their combined variability still creates operational challenges related to power imbalance, renewable curtailment, battery degradation, peak demand management, grid stability, and electricity market participation. Conventional rule-based and optimization-based scheduling strategies often depend on accurate forecasts and fixed operating rules, which may limit their ability to adapt to rapidly changing renewable generation, load demand, electricity prices, and battery state conditions. To address these challenges, this paper proposes a Deep Q-Network (DQN) reinforcement learning framework for real-time energy storage scheduling in wind–solar hybrid systems.

The proposed framework formulates storage scheduling as a sequential decision-making problem in which a DQN agent learns an optimal control policy through continuous interaction with the hybrid energy environment. The system state includes photovoltaic generation, wind power output, load demand, battery state of charge, electricity price, weather forecast indicators, grid import/export status, and previous scheduling actions. Based on these observations, the DQN agent selects discrete energy storage actions, including charging, discharging, idle operation, grid import support, renewable curtailment reduction, and peak shaving decisions. The reward function is designed to minimize total operating cost, renewable energy curtailment, unmet load, battery degradation, and grid power fluctuation while maximizing renewable energy utilization and system reliability.

The proposed model incorporates key components of Deep Q-Network learning, including experience replay, target network stabilization, epsilon-greedy exploration, and temporal-difference learning. These mechanisms enable the agent to learn stable and efficient scheduling policies under uncertain and nonlinear operating conditions. A comprehensive simulation environment is developed using historical solar irradiance, wind speed, load demand, electricity tariff, and battery performance data. Data preprocessing includes missing value imputation, outlier removal, normalization, temporal alignment, scenario generation, and construction of realistic renewable generation profiles. Battery operational constraints, including state-of-charge limits, charging/discharging efficiency, power capacity, depth of discharge, and cycle degradation cost, are incorporated to ensure physically feasible scheduling decisions.

The proposed DQN-based scheduling system can be evaluated under multiple operating scenarios, including grid-connected operation, islanded microgrid operation, high renewable penetration, peak load periods, uncertain weather conditions, and time-of-use electricity pricing. Performance is assessed using total energy cost, renewable utilization rate, curtailment reduction, battery degradation cost, load satisfaction ratio, peak-to-average ratio, grid power exchange variability, convergence behavior, and computational response time. Comparative analysis is conducted against rule-based scheduling, mixed-integer linear programming, dynamic programming, model predictive control, Q-learning, and heuristic energy management strategies. Experimental evaluation is expected to demonstrate that the proposed DQN framework achieves more adaptive and cost-effective storage scheduling by learning from real-time system feedback rather than relying solely on predefined control rules.

To support practical deployment, the framework can be integrated into smart grid controllers, microgrid energy management systems, and distributed energy resource management platforms. The learned policy enables real-time decision-making for battery charging and discharging while considering renewable uncertainty, load variability, and operational constraints. Furthermore, interpretability analysis is incorporated through reward decomposition, action-value visualization, and state importance analysis to help operators understand why specific storage actions are selected under different system conditions. These insights improve trust and facilitate operational validation before deployment in real-world hybrid renewable systems.

The proposed DQN reinforcement learning framework provides a scalable, adaptive, and intelligent solution for real-time energy storage scheduling in wind–solar hybrid systems. By combining deep reinforcement learning with battery-aware operational modeling, the system can reduce energy cost, improve renewable energy utilization, limit curtailment, enhance grid stability, and support the transition toward more resilient and sustainable power systems.

 

Keywords

References

  • [1] Abdollahzadeh, B., Khodadadi, N., Barshandeh, S., Trojovský, P., Gharehchopogh, F. S., El -kenawy, E. S. M., ... & Mirjalili, S. (2024).
  • [2] El-Kenawy, E. S. M., Ibrahim, A., Mirjalili, S., Eid, M. M., & Hussein, S. E. (2020). Novel feature selection and voting classif ier
  • [3] El-Kenawy, E. S. M., Eid, M. M., Saber, M., & Ibrahim, A. (2020). MbGWO -SFS: Modified binary grey wolf optimizer based on
  • [4] El-Kenawy, E. S., & Eid, M. (2020). Hybrid gray wolf and particle swarm optimization for feature selection. Int. J. Innov. Compu t. Inf.
  • [5] El-Kenawy, E. S. M., Mirjalili, S., Ibrahim, A., Alrahmawy, M., El -Said, M., Zaki, R. M., & Eid, M. M. (2021). Advanced meta -
  • [6] Khodadadi, N., Abualigah, L., El -Kenawy, E. S. M., Snasel, V., & Mirjalili, S. (2022). An archive -based multi -objective arithmetic
  • [7] Ibrahim, A., Mirjalili, S., El -Said, M., Ghoneim, S. S., Al -Harthi, M. M., Ibrahim, T. F., & El -Kenawy, E. S. M. (2021). Wind speed
  • [8] Abdelhamid, A. A., El -Kenawy, E. S. M., Alotaibi, B., Amer, G. M., Abdelkader, M. Y., Ibrahim, A., & Eid, M. M. (2022). Robust
  • [9] El-Kenawy, E. S. M., Mirjalili, S., Alassery, F., Zhang, Y. D., Eid, M. M., El -Mashad, S. Y., ... & Abdelhamid, A. A. (2022). Novel
  • [10] Hassib, E. M., El -Desouky , A. I., Labib, L. M., & El -Kenawy, E. S. M. (2020). WOA+ BRNN: An imbalanced big data classification
  • [11] Eid, M. M., El -kenawy, E. S. M., & Ibrahim, A. (2021, March). A binary sine cosine -modified whale optimization algorithm for feature
  • [12] El-Sayed Towfek, M. (2018). El -kenawy. Trust Model for Dependable File Exchange in Cloud Computing. International Journal of
  • [13] Abdelhamid, A. A., Towfek, S. K., Khodadadi, N., Alhussan, A. A., Khafaga, D. S., Eid, M. M., & Ibrahim, A. (2023). Waterwhee l plant
  • [14] El-Kenawy, E. S. M., Abdelhamid, A. A., Ibrahim, A., Mirjalili, S., Khodadad, N., Alduailij, M. A., ... & Khafaga, D. S. (2023). Al-
  • [15] Alhussan, A. A., Abdelhamid, A. A., El -Kenawy, E. S. M., Ibrahim, A., Eid, M. M., Khafaga, D. S., & Ahmed, A. E. (2023). A binary
  • [16] Alhussan, A. A., Khafaga, D. S., Abotaleb, M., Mishra, P., & El -Kenawy, E. S. M. (2024). Global Potato Production Forecasting Based
  • [17] Towfek, S. K., & Alhussan, A. A. (2024). Potato Production Forecasting Based on Balance Dynamic Biruni Earth Radius Algorithm for
  • [18] Mishra, P., Alhussan, A. A., Khafaga, D. S., Lal, P., Ray, S., Abotaleb, M., ... & El -Kenawy, E. S. M. (2024). Forecasting production of
  • [19] El-Kenawy, E. S. M., Mirjalili, S., Abdelhamid, A. A., Ibrahim, A., Khodadadi, N., & Eid, M. M. (2022). Meta -heuristic optimization
  • [20] Abdelhamid, A. A., El -Kenawy, E. S. M., Khodadadi, N., Mirjalili, S., Khafaga, D. S., Alharbi, A. H., ... & Saber, M. (2022).
  • [21] Alharbi, A. H., Towfek, S. K., Abdelhamid, A. A., Ibrahim, A., Eid, M. M., & Khafaga, D. S. & Saber, M.(2023). Diagnosis of
  • [22] Alhussan, A. A., & Towfek, S. K. (2024). 5G Resource Allocation Using Feature Selection and Greylag Goose Optimization Algori thm.
  • [23] Towfek, S. K., & Alhussan, A. A. (2024). Potato Production Forecasting Based on Balance Dynamic Biruni Earth Radius Algorithm for
  • [24] Abdelhamid, A. A., Alhussan, A. A., Qenawy, A. S. T., Osman, A. M., Elshewey, A. M., & Eed, M. (2024). Potato harvesting pred iction
  • [25] Eed, M., Alhussan, A. A., Qenawy, A. S. T., Osman, A. M., Elshewey, A. M., & Arnous, R. (2024). Potato Consumption Forecastin g
  • [26] Mahmood, S., Sun, H., Iqbal, A., Alhussan, A. A., & El -kenawy, E. S. M. (2024). Green finance, sustainable infrastructure, and green
  • [27] El-Kenawy, E. S. M., Alhussan, A. A., Khodadadi, N., Mirjalili, S., & Eid, M. M. (2024). Predicting potato crop yield with machi ne
  • [28] Radwan, M., Alhussan, A. A., Ibrahim, A., & Tawfeek, S. M. (2025). Potato leaf disease classification using optimized machine learning
  • [29] Ghasemi, M., Khodadadi, N., Trojovský, P., Li, L., Mansor, Z., Abualigah, L., ... & El -Kenawy, E. S. M. (2025). Kirchhoff’s law
  • [30] Yassen, M. A., El -Kenawy, E. S. M., Abdel -Fattah, M. G., Ismail, I., & Mostafa, H. E. D. S. (2025). Explainable artificial intelligence
  • [31] Radwan, M., Alhussan, A. A., Ibrahim, A., & Tawfeek, S. M. (2025). Potato leaf disease classification using optimized machine learning
  • [32] Mozhdehi, A. T., Khodadadi, N., Aboutalebi, M., El -Kenawy, E. S. M., Hussien, A. G., Zhao, W., ... & Mirjalili, S. (2025). Divine
  • [33] El-kenawy, E. S. M., Alhussan, A. A., Mattar, E. A., & Radwan, M. (2026). Feature selection and hyperparameter tuning in transfo rmer -
  • [34] Chen, L., Xu, C., Lim, W. H., Sharma, A., Tiang, S. S., Chong, K. S., ... & Khafaga, D. S. (2025). Transparent and reliable c onstruction
  • [35] Khodadadi, N., Towfek, S. K., Zaki, A. M., Alharbi, A. H., Khodadadi, E., Khafaga, D. S., ... & Eid, M. M. (2025). Predicting
  • [36] Alhussan, A. A., El -Kenawy, E. S. M., Khafaga, D. S., Alharbi, A. H., & Eid, M. M. (2025). Groundwater resource prediction and
  • [37] Mozhdehi, A. T., Khodadadi, N., Aboutalebi , M., El -Kenawy, E. S. M., Hussien, A. G., Zhao, W., ... & Mirjalili, S. (2025). Divine
  • [38] Lopes, J. M., Pinho, C. S., Martins, R. C., & Bandeira, T. (2026). Sustainability Gets Smarter: Competitive Pressure and AI L eading the
  • [39] Radwan, M., Ibrahim, A., Abdelsalam, M. M., Alhussan, A. A., Mattar, E. A., & El -Kenawy, E. S. M. (2026). Optimizing solar and wind
  • [40] Alhussan, A. A., El -Kenawy, E. S. M., Eid, M. M., & Khodadadi, N. (2026). Hybrid Al -Biruni and Puma Optimization (BERPO) for
  • [41] El-Kenawy, E. S. M., Ibrahim, A., Alhussan, A. A., Khafaga, D. S., Ahmed, A. E., & Eid, M. M. (2026). Smart city electricity loa d
  • [42] El-Kenawy, E. S. M., Khodadadi, N., Mirjalili, S., Zaki, A. M., Ibrahim, A., Alhussan, A. A., ... & Eid, M. M. (2026). Glider sn ake
  • [43] Mekaret, F., Rabehi, A., Zebentout, B., Tizi, S., Douara, A., Bellucci, S., ... & Alhussan, A. A. (2024). A comparative study of Schottky
  • [44] KOUADRI, Ali, RABEHI, Abdelhalim, BENZIANE, Ali, et al. A Robust Multi -Transform Watermarking Scheme for Medical Images
  • [45] Tibermacine, I. E., Russo, S., Scarano, G., Tedesco, G., Rabehi, A., Alhussan, A. A., ... & Napoli, C. (2025). Conditional VA E for
  • [46] Kouadri, A., Benziane, A., Rabehi, A., Rabehi, A., Alhussan, A. A., Khafaga, D. S., & El -Kenawy, E. S. M. (2025). A novel hybrid
  • [47] Russo, S., Tibermacine, I. E., Randieri, C., Rabehi, A., Alharbi, A. H., El -Kenawy, E. S. M., & Napoli, C. (2025). Exploiting facial
  • [48] Bentegri, H., Rabehi, M., Kherfane, S., Nahool, T. A., Rabehi, A., Guermoui, M., ... & El -Kenawy, E. S. M. (2025). Assessment of
  • [49] Tibermacine, I. E., Russo, S., Citeroni, F., Mancini, G., Rabehi, A., Alharbi, A. H., ... & Napoli, C. (2025). Adversarial de noising of EEG
  • [50] Ouahabi, M. S., Benyounes, A., Barkat, S., Ihammouchen, S., Rekioua, T., Rabehi, A., ... & Alharbi, A. H. (2025). Real -time sensor fault
  • [51] Belaid, A., Guermoui, M., Khelifi, R., Arrif, T., Chekifi, T., Rabehi, A., ... & Alhussan, A. A. (2024). Assessing suitable a reas for PV
  • [52] Mehallou, A., M’hamdi, B., Amari, A., Teguar, M., Rabehi, A., Guermoui, M., ... & Khafaga, D. S. (2025). Optimal multiobjecti ve
  • [53] Rabehi, A., El -Hadi, M., Benmahmoud, S., Rabehi, A., Alharbi, A. H., & El -Kenawy, E. S. M. (2025). SOCA -CFAR Processor in A
  • [54] Bakria, D., Beladel, A., Korich, B., Teta, A., Mohammedi, R. D., Laouid, A. A., ... & El -kenawy, E. S. (2025). A novel enhanced Grey
Download