Skip to main content
Springer Nature Link
Log in

Deep reinforcement learning-based network for optimized power flow in islanded DC microgrid

  • Original Paper
  • Published:
Electrical Engineering Aims and scope Submit manuscript

Abstract

This paper presents an optimum power flow control for islanded microgrid employing deep reinforcement learning. During abnormal grid conditions, the stability of the microgrids is very important to avoid grid outages. In abnormal grid condition, the microgrid operates in the islanded mode for providing uninterrupted supply to loads and stability improvement with power resilience. This islanded operation depends on the effective operation of connected Distributed Renewable Energy Sources (DRES). This paper aims to provide optimum power dispatch, and accurate control of connected DRES enables the grid to restore service. Advanced data-driven control method could provide a solution to grid outages and DRES service support. In this research work, a Deep Reinforcement Learning Network (DRLN) was proposed to identify the State of Charge (SoC) for optimum power flow during source outages, communication link failure, and communication bus failure. A finite horizon policy update provides updated DRES SoC and makes the grid to operate on the islanded mode of operation with service. This provides an optimized power flow and accurate performance. By analysing the voltage, current, and weighted average power trajectories, the performance of the proposed DRLN was validated. The simulation was studied with a constant ZIP load, and also the proposed algorithm was implemented in IEEE 15 bus system to demonstrate its effectiveness. Finally, Hardware in Loop simulation is implemented to validate the results of the proposed DRLN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Availability of data and materials

The data presented in this study are available by reasonable request to the corresponding author.

Code availability

The deep learning code presented in this study is available by reasonable request to the corresponding author.

References

  1. Abdolaziz Mohamed M, Almalaq A, Mahrous Awwad E, El-Meligy MA, Sharaf M, Ali ZM (2020) A modified balancing approach for renewable based microgrids using deep adversarial learning. IEEE Trans Ind Appl 9994(c):1–1. https://doi.org/10.1109/tia.2020.3002187

    Article  Google Scholar 

  2. Alavi SA, Mehran K, Vahidinasab V, Catalao JPS (2021) Forecast-based consensus control for DC microgrids using distributed long short-term memory deep learning models. IEEE Trans Smart Grid 12(5):3718–3730. https://doi.org/10.1109/TSG.2021.3070959

    Article  Google Scholar 

  3. Bui VH, Hussain A, Kim HM (2020) Double deep Q -learning-based distributed operation of battery energy storage system considering uncertainties. IEEE Trans Smart Grid 11(1):457–469. https://doi.org/10.1109/TSG.2019.2924025

    Article  Google Scholar 

  4. Cao D, Hu W, Xu X, Wu Q, Huang Q, Chen Z, Blaabjerg F (2021) Deep reinforcement learning based approach for optimal power flow of distribution networks embedded with renewable energy and storage devices. J Mod Power Syst Clean Energy 9(5):1101–1110. https://doi.org/10.35833/MPCE.2020.000557

    Article  Google Scholar 

  5. Cao D, Zhao J, Hu W, Ding F, Huang Q, Chen Z, Blaabjerg F (2021) Data-driven multi-agent deep reinforcement learning for distribution system decentralized voltage control with high penetration of PVs. IEEE Trans Smart Grid 12(5):4137–4150. https://doi.org/10.1109/TSG.2021.3072251

    Article  Google Scholar 

  6. Dabbaghjamanesh M, Kavousi-Fard A, Zhang J (2021) Stochastic modeling and integration of plug-in hybrid electric vehicles in reconfigurable microgrids with deep learning-based forecasting. IEEE Trans Intell Transp Syst 22(7):4394–4403. https://doi.org/10.1109/TITS.2020.2973532

    Article  Google Scholar 

  7. Du Y, Li F (2020) Intelligent multi-microgrid energy management based on deep neural network and model-free reinforcement learning. IEEE Trans Smart Grid 11(2):1066–1076. https://doi.org/10.1109/TSG.2019.2930299

    Article  Google Scholar 

  8. Du Y, Wu D (2022) Deep reinforcement learning from demonstrations to assist service restoration in islanded microgrids. IEEE Trans Sustain Energy 3029(c):1–11. https://doi.org/10.1109/TSTE.2022.3148236

    Article  Google Scholar 

  9. Fan B, Peng J, Yang Q, Liu W (2021) Distributed control of DC microgrids with improved ZIP load adaptability. IEEE Trans Syst Man Cybern Syst. https://doi.org/10.1109/tsmc.2021.3101813

    Article  Google Scholar 

  10. Gao S, Xiang C, Yu M, Tan KT, Lee TH (2022) Online optimal power scheduling of a microgrid via imitation learning. IEEE Trans Smart Grid 13(2):861–876. https://doi.org/10.1109/TSG.2021.3122570

    Article  Google Scholar 

  11. Jeyaraj PR, Asokan SP, Karthiresan AC (2022) Optimum power flow in DC microgrid employing Bayesian regularized deep neural network. Electr Power Syst Res 205(December 2021):107730. https://doi.org/10.1016/j.epsr.2021.107730

    Article  Google Scholar 

  12. Jeyaraj PR, Kathiresan AC, Asokan SP, Nadar ERS, Rezk H, Babu TS (2021) Power system resiliency and wide area control employing deep learning algorithm. Comput Mater Contin 68(1):553–567. https://doi.org/10.32604/cmc.2021.015128

    Article  Google Scholar 

  13. Lei L, Tan Y, Dahlenburg G, Xiang W, Zheng K (2021) Dynamic energy dispatch based on deep reinforcement learning in IoT-driven smart isolated microgrids. IEEE Internet Things J 8(10):7938–7953. https://doi.org/10.1109/JIOT.2020.3042007

    Article  Google Scholar 

  14. Lei X, Yang Z, Yu J, Zhao J, Gao Q, Yu H (2021) Data-driven optimal power flow: a physics-informed machine learning approach. IEEE Trans Power Syst 36(1):346–354. https://doi.org/10.1109/TPWRS.2020.3001919

    Article  Google Scholar 

  15. Lu Y, Liang Y, Ding Z, Wu Q, Ding T, Lee WJ (2022) Deep reinforcement learning-based charging pricing for autonomous mobility-on-demand system. IEEE Trans Smart Grid 13(2):1412–1426. https://doi.org/10.1109/TSG.2021.3131804

    Article  Google Scholar 

  16. Mahfuz-Ur-Rahman AM, Islam MR, Muttaqi KM, Sutanto D (2021) An effective energy management with advanced converter and control for a PV-battery storage based microgrid to improve energy resiliency. IEEE Trans Ind Appl 57(6):6659–6668. https://doi.org/10.1109/TIA.2021.3115085

    Article  Google Scholar 

  17. Patarroyo-Montenegro JF, Andrade F, Guerrero JM, Vasquez JC (2021) A linear quadratic regulator with optimal reference tracking for three-phase inverter-based islanded microgrids. IEEE Trans Power Electron 36(6):7112–7122. https://doi.org/10.1109/TPEL.2020.3036594

    Article  Google Scholar 

  18. Su T, Liu Y, Zhao J, Liu J (2022) Modeling for Fast Preventive Control of Power 18(1):315–326

  19. Velloso A, Van Hentenryck P (2021) Combining deep learning and optimization for preventive security-constrained DC optimal power flow. IEEE Trans Power Syst 36(4):3618–3628. https://doi.org/10.1109/TPWRS.2021.3054341

    Article  Google Scholar 

  20. Xie P, Guerrero JM, Tan S, Bazmohammadi N, Vasquez JC, Mehrzadi M, Al-Turki Y (2021) Optimization-based power and energy management system in shipboard microgrid: a review. IEEE Syst J. https://doi.org/10.1109/JSYST.2020.3047673

    Article  Google Scholar 

  21. Xu Y, Yan R, Wang Y, Jiahong D (2022) A multi-agent quantum deep reinforcement learning method for distributed frequency control of islanded microgrids. IEEE Trans Control Network Syst. https://doi.org/10.1109/TCNS.2022.3140702

    Article  Google Scholar 

  22. Yuan X, Han P, Duan Y, Alden RE, Rallabandi V, Ionel DM (2020) Residential electrical load monitoring and modeling-state of the art and future trends for smart homes and grids. Electr Power Compon Syst 48(11):1125–1143. https://doi.org/10.1080/15325008.2020.1834019

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to express their profound gratitude to Management, Principal and Department of Electrical and Electronics Engineering of Mepco Schlenk Engineering College (Autonomous), Sivakasi, Tamil Nadu for providing us the necessary facilities to carry out our research work. This research work is supported by DST-SERB project under TARE scheme (TAR/2021/000396). Also, the authors like to express the profound gratitude to the respected reviewers for giving necessary suggestion for manuscript’s constructive improvement.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronics Engineering, Mepco Schlenk Engineering College (Autonomous), Sivakasi, India

    Pandia Rajan Jeyaraj, Siva Prakash Asokan, Aravind Chellachi Kathiresan & Edward Rajan Samuel Nadar

Authors
  1. Pandia Rajan Jeyaraj
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Siva Prakash Asokan
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Aravind Chellachi Kathiresan
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Edward Rajan Samuel Nadar
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Pandia Rajan Jeyaraj: Conceptualization, Methodology, Validation, Data Curation, Formal analysis, Software, Writing - Original Draft, Writing- Reviewing and Editing. •Siva Prakash Asokan: Validation, Writing- Reviewing and Editing. •Aravind Chellachi Kathiresan: Investigation, Supervision. •Edward Rajan Samuel Nadar: Supervision, Project administration.

Corresponding author

Correspondence to Pandia Rajan Jeyaraj.

Ethics declarations

Conflict of interest

All Authors declare that they have No Potential conflicts of interest.

Human and animal rights

In this research, No Human Participants and Animals are involved.

Informed consent

No Informed consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jeyaraj, P.R., Asokan, S.P., Kathiresan, A.C. et al. Deep reinforcement learning-based network for optimized power flow in islanded DC microgrid. Electr Eng 105, 2805–2816 (2023). https://doi.org/10.1007/s00202-023-01835-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00202-023-01835-1

Keywords