International Journal of Computer Science & Network Security

국제컴퓨터통신보호논문지학회 (International Journal of Computer Science & Network Security)
권호 표지
DOI QR코드

DOI QR Code

The Development of an Intelligent Home Energy Management System Integrated with a Vehicle-to-Home Unit using a Reinforcement Learning Approach

  • Ohoud Almughram (Department of Information Systems, College of Computer Science, King Khalid University) ;
  • Sami Ben Slama (Analysis and Processing of Electrical and Energy Systems Unit, Faculty of Sciences of Tunis El Manar) ;
  • Bassam Zafar (Faculty of Computing and Information Technology, King Abdulaziz University)
  • 투고 : 2024.04.05
  • 발행 : 2024.04.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Vehicle-to-Home (V2H) and Home Centralized Photovoltaic (HCPV) systems can address various energy storage issues and enhance demand response programs. Renewable energy, such as solar energy and wind turbines, address the energy gap. However, no energy management system is currently available to regulate the uncertainty of renewable energy sources, electric vehicles, and appliance consumption within a smart microgrid. Therefore, this study investigated the impact of solar photovoltaic (PV) panels, electric vehicles, and Micro-Grid (MG) storage on maximum solar radiation hours. Several Deep Learning (DL) algorithms were applied to account for the uncertainty. Moreover, a Reinforcement Learning HCPV (RL-HCPV) algorithm was created for efficient real-time energy scheduling decisions. The proposed algorithm managed the energy demand between PV solar energy generation and vehicle energy storage. RL-HCPV was modeled according to several constraints to meet household electricity demands in sunny and cloudy weather. Simulations demonstrated how the proposed RL-HCPV system could efficiently handle the demand response and how V2H can help to smooth the appliance load profile and reduce power consumption costs with sustainable power generation. The results demonstrated the advantages of utilizing RL and V2H as potential storage technology for smart buildings.

키워드

참고문헌

  1. J. Corral-Garcia, F. Lemus-Prieto, J.-L. Gonzalez-Sanchez, and M.-A. Perez-Toledano, "Analysis of Energy Consumption and Optimization Techniques for Writing Energy-Efficient Code," Electronics (Basel), vol. 8, no. 10, p. 1192, 2019.
  2. M. S. Aliero, M. Asif, I. Ghani, M. F. Pasha, and S. R. Jeong, "Systematic Review Analysis on Smart Building: Challenges and Opportunities," Sustainability, vol. 14, no. 5, p. 3009, 2022.
  3. G. Hafeez et al., "Efficient Energy Management of IoT-Enabled Smart Homes Under Price-Based Demand Response Program in Smart Grid," Sensors, vol. 20, no. 11, p. 3155, 2020.
  4. J. S. GK, "MANFIS based SMART home energy management system to support SMART grid.," Peer Peer Netw Appl, vol. 13, no. 6, 2020.
  5. B. Sharma and J. K. Maherchandani, "A Review on Integration of Electric Vehicle in Smart Grid: Operational modes, Issues and Challenges," in 2022 International Conference on Computer Communication and Informatics (ICCCI), 2022, pp. 1-5.
  6. N. S. Pearre and H. Ribberink, "Review of research on V2X technologies, strategies, and operations," Renewable and Sustainable Energy Reviews, vol. 105, pp. 61-70, 2019. https://doi.org/10.1016/j.rser.2019.01.047
  7. S.-H. Park, A. Hussain, and H.-M. Kim, "Impact analysis of survivability-oriented demand response on islanded operation of networked microgrids with high penetration of renewables," Energies (Basel), vol. 12, no. 3, p. 452, 2019.
  8. F. Alfaverh, M. Denai, and Y. Sun, "Electrical vehicle grid integration for demand response in distribution networks using reinforcement learning," IET Electrical Systems in Transportation, vol. 11, no. 4, pp. 348-361, 2021. https://doi.org/10.1049/els2.12030
  9. U. ur Rehman, K. Yaqoob, and M. A. Khan, "Optimal power management framework for smart homes using electric vehicles and energy storage," International Journal of Electrical Power & Energy Systems, vol. 134, p. 107358, 2022.
  10. M. H. Elkholy, H. Metwally, M. A. Farahat, M. Nasser, T. Senjyu, and M. E. Lotfy, "Dynamic centralized control and intelligent load management system of a remote residential building with V2H technology," J Energy Storage, vol. 52, p. 104839, 2022.
  11. G. S. Georgiou, P. Christodoulides, and S. A. Kalogirou, "Optimizing the energy storage schedule of a battery in a PV grid-connected nZEB using linear programming," Energy, vol. 208, p. 118177, 2020.
  12. S. S. R. Nasab, A. T. Nasrabadi, S. Asadi, and S. A. H. S. Taghia, "Investigating the probability of designing net-zero energy buildings with consideration of electric vehicles and renewable energy," Engineering, Construction and Architectural Management, 2021.
  13. S. Rout and G. R. Biswal, "Bidirectional EV integration in home load energy management using swarm intelligence," International Journal of Ambient Energy, pp. 1-8, 2022.
  14. A. Dileep and B. M. Beena, "Global Distribution and Price Prediction of Electric Vehicles Using Machine Learning," in 2022 2nd International Conference on Intelligent Technologies (CONIT), 2022, pp. 1-6.
  15. D. Aguilar-Dominguez, J. Ejeh, A. D. F. Dunbar, and S. F. Brown, "Machine learning approach for electric vehicle availability forecast to provide vehicle-to-home services," Energy Reports, vol. 7, pp. 71-80, 2021. https://doi.org/10.1016/j.egyr.2021.02.053
  16. R. Khezri and D. Steen, "Vehicle to Everything (V2X)-A Survey on Standards and Operational Strategies," in 2022 IEEE International Conference on Environment and Electrical Engineering and 2022 IEEE Industrial and Commercial Power Systems Europe (EEEIC/I&CPS Europe), 2022, pp. 1-6.
  17. X. Xu, Y. Jia, Y. Xu, Z. Xu, S. Chai, and C. S. Lai, "A multi-agent reinforcement learning-based data-driven method for home energy management," IEEE Trans Smart Grid, vol. 11, no. 4, pp. 3201-3211, 2020. https://doi.org/10.1109/TSG.2020.2971427
  18. US Energy Information Administration, "Independent Statistics and Analysis," 2021. https://www.eia.gov/totalenergy/data/annual/
  19. N. S. Pearre and H. Ribberink, "Review of research on V2X technologies, strategies, and operations," Renewable and Sustainable Energy Reviews, vol. 105, pp. 61-70, 2019. https://doi.org/10.1016/j.rser.2019.01.047
  20. C. Gschwendtner, S. R. Sinsel, and A. Stephan, "Vehicle-to-X (V2X) implementation: An overview of predominate trial configurations and technical, social and regulatory challenges," Renewable and Sustainable Energy Reviews, vol. 145, p. 110977, 2021.
  21. A. Ahmadian, B. Mohammadi-Ivatloo, and A. Elkamel, "A review on plug-in electric vehicles: introduction, current status, and load modeling techniques," Journal of Modern Power Systems and Clean Energy, vol. 8, no. 3, pp. 412-425, 2020. https://doi.org/10.35833/MPCE.2018.000802
  22. S. Vadi, R. Bayindir, A. M. Colak, and E. Hossain, "A review on communication standards and charging topologies of V2G and V2H operation strategies," Energies (Basel), vol. 12, no. 19, p. 3748, 2019.
  23. A. Chatzivasileiadi, E. Ampatzi, and I. P. Knight, "Electrical energy storage sizing and space requirements for sub-daily autonomy in residential buildings," Energies (Basel), vol. 15, no. 3, p. 1145, 2022.
  24. E. Solomon, I. G. Hagos, and B. Khan, "Outage management in a smart distribution grid integrated with PEV and its V2B applications," in Active Electrical Distribution Network, Elsevier, 2022, pp. 217-226.
  25. S. Das et al., "Toyota smarthome: Real-world activities of daily living," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 833-842.
  26. B. Cui, S. Lee, P. Im, M. Koenig, and M. Bhandari, "High resolution dataset from a net-zero home that demonstrates zero-carbon living and transportation capacity," Data Brief, vol. 45, p. 108703, 2022.
  27. H. Mehrjerdi, "Resilience oriented vehicle-to-home operation based on battery swapping mechanism," Energy, vol. 218, p. 119528, 2021.
  28. M. S. Shamami et al., "Artificial Intelligence-Based Performance Optimization of Electric Vehicle-to-Home (V2H) Energy Management System," SAE International Journal of Sustainable Transportation, Energy, Environment, & Policy, vol. 1, no. 13-01-02-0007, 2020.
  29. S. A. ben Slama, "Design and implementation of home energy management system using vehicle to home (H2V) approach," J Clean Prod, p. 127792, 2021.
  30. R. S. Sundararajan and M. T. Iqbal, "Dynamic Modelling of a Solar Energy System with Vehicle to Home Option for Newfoundland Conditions," European Journal of Engineering and Technology Research, vol. 6, no. 5, pp. 16-23, 2021. https://doi.org/10.24018/ejeng.2021.6.5.2497
  31. T. Higashitani, T. Ikegami, A. Uemichi, and A. Akisawa, "Evaluation of residential power supply by photovoltaics and electric vehicles," Renew Energy, vol. 178, pp. 745-756, 2021.
  32. H. Nakano et al., "Aggregation of V2H systems to participate in regulation market," IEEE Transactions on Automation Science and Engineering, vol. 18, no. 2, pp. 668-680, 2020. https://doi.org/10.1109/TASE.2020.3001060
  33. M. Kosinka, Z. Slanina, M. Petruzela, and V. Blazek, "V2H control system software analysis and design," in 2020 20th International Conference on Control, Automation and Systems (ICCAS), 2020, pp. 972-977.
  34. O. Ouramdane, E. Elbouchikhi, Y. Amirat, F. le Gall, and E. Sedgh Gooya, "Home Energy Management Considering Renewable Resources, Energy Storage, and an Electric Vehicle as a Backup," Energies (Basel), vol. 15, no. 8, p. 2830, 2022.
  35. O. Vermesan et al., "Automotive intelligence embedded in electric connected autonomous and shared vehicles technology for sustainable green mobility," Frontiers in Future Transportation, vol. 2, p. 688482, 2021.
  36. S. Shahriar, A. Al-Ali, A. Osman, S. Dhou, and M. Nijim, "Machine Learning Approaches for EV Charging Behavior: A Review," IEEE Access, vol. 8, pp. 168980-168993, Dec. 2020, doi: 10.1109/ACCESS.2020.3023388.
  37. Y. Yang and S. Wang, "Resilient residential energy management with vehicle-to-home and photovoltaic uncertainty," International Journal of Electrical Power & Energy Systems, vol. 132, p. 107206, 2021.
  38. D. Borge-Diez, D. Icaza, E. Acikkalp, and H. Amaris, "Combined vehicle to building (V2B) and vehicle to home (V2H) strategy to increase electric vehicle market share," Energy, vol. 237, p. 121608, 2021.
  39. A. Mohammad et al., "Integration of electric vehicles and energy storage system in home energy management system with home to grid capability," Energies (Basel), vol. 14, no. 24, p. 8557, 2021.
  40. Y. Ye, D. Qiu, H. Wang, Y. Tang, and G. Strbac, "Real-time autonomous residential demand response management based on twin delayed deep deterministic policy gradient learning," Energies (Basel), vol. 14, no. 3, p. 531, 2021.
  41. R. Kelm Paweland Mienski and I. Wasiak, "Energy Management in a Prosumer Installation Using Hybrid Systems Combining EV and Stationary Storages and Renewable Power Sources," Applied Sciences, vol. 11, no. 11, p. 5003, 2021.
  42. S. Rafique, M. J. Hossain, M. S. H. Nizami, U. bin Irshad, and S. C. Mukhopadhyay, "Energy management systems for residential buildings with electric vehicles and distributed energy resources," IEEE Access, vol. 9, pp. 46997-47007, 2021. https://doi.org/10.1109/ACCESS.2021.3067950
  43. A. T. Eseye, M. Lehtonen, T. Tukia, S. Uimonen, and R. J. Millar, "Optimal energy trading for renewable energy integrated building microgrids containing electric vehicles and energy storage batteries," IEEE Access, vol. 7, pp. 106092-106101, 2019. https://doi.org/10.1109/ACCESS.2019.2932461
  44. S. Lee, H. Jin, L. F. Vecchietti, J. Hong, and D. Har, "Short-term predictive power management of PV-powered nanogrids," Ieee Access, vol. 8, pp. 147839-147857, 2020. https://doi.org/10.1109/ACCESS.2020.3015243
  45. M. Yousefi, A. Hajizadeh, M. N. Soltani, and B. Hredzak, "Predictive home energy management system with photovoltaic array, heat pump, and plug-in electric vehicle," IEEE Trans Industr Inform, vol. 17, no. 1, pp. 430-440, 2020.
  46. A. C. Duman, H. S. Erden, O. Gonul, and O. Guler, "A home energy management system with an integrated smart thermostat for demand response in smart grids," Sustain Cities Soc, vol. 65, p. 102639, 2021.
  47. X. Liu, Y. Wu, and H. Wu, "PV-EV Integrated Home Energy Management Considering Residential Occupant Behaviors," Sustainability, vol. 13, no. 24, p. 13826, 2021.
  48. A. T. Tran, A. Kawashima, S. Inagaki, and T. Suzuki, "Design of a Home Energy Management System Integrated with an Electric Vehicle (V2H+HPWH EMS)," in Design and Analysis of Distributed Energy Management Systems, Springer, 2020, pp. 47-66.
  49. S. Lee and D.-H. Choi, "Reinforcement learning-based energy management of smart home with rooftop solar photovoltaic system, energy storage system, and home appliances," Sensors, vol. 19, no. 18, p. 3937, 2019.
  50. N. Ashenov, M. Myrzaliyeva, M. Mussakhanova, and H. K. Nunna, "Dynamic Cloud and ANN based Home Energy Management System for End-Users with Smart-Plugs and PV Generation," in 2021 IEEE Texas Power and Energy Conference (TPEC), 2021, pp. 1-6.
  51. M. Kim, H. Kim, and J. H. Jung, "A Study of Developing a Prediction Equation of Electricity Energy Output via Photovoltaic Modules," Energies (Basel), vol. 14, no. 5, p. 1503, 2021.
  52. A. Vulkan, I. Kloog, M. Dorman, and E. Erell, "Modeling the potential for PV installation in residential buildings in dense urban areas," Energy Build, vol. 169, pp. 97-109, 2018. https://doi.org/10.1016/j.enbuild.2018.03.052
  53. H. A. Muqeet et al., "IoT-based intelligent source-load-storage coordination scheme for prosumer campus microgrids," Front Energy Res, vol. 10, 2022.
  54. A. Khalid, N. Javaid, A. Mateen, M. Ilahi, T. Saba, and A. Rehman, "Enhanced time-of-use electricity price rate using game theory," Electronics (Basel), vol. 8, no. 1, p. 48, 2019.
  55. S. Aslam, "An optimal home energy management scheme considering grid connected microgrids with day-ahead weather forecasting using artificial neural network," Masters Thesis, 2018.
  56. M. Rafiei, T. Niknam, J. Aghaei, M. Shafie-Khah, and J. P. S. Catalao, "Probabilistic load forecasting using an improved wavelet neural network trained by generalized extreme learning machine," IEEE Trans Smart Grid, vol. 9, no. 6, pp. 6961-6971, 2018. https://doi.org/10.1109/TSG.2018.2807845
  57. S. Chai, Z. Xu, and Y. Jia, "Conditional density forecast of electricity price based on ensemble ELM and logistic EMOS," IEEE Trans Smart Grid, vol. 10, no. 3, pp. 3031-3043, 2018.
  58. W. Fu, K. Wang, C. Li, and J. Tan, "Multi-step short-term wind speed forecasting approach based on multi-scale dominant ingredient chaotic analysis, improved hybrid GWO-SCA optimization and ELM," Energy Convers Manag, vol. 187, pp. 356-377, 2019.
  59. H. R. Berenji, "Fuzzy Q-learning: a new approach for fuzzy dynamic programming," in Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference, 1994, pp. 486-491.
  60. H. R. Berenji, "Fuzzy Q-learning: a new approach for fuzzy dynamic programming," in Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference, 1994, pp. 486-491.