IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

Obstacle Avoidance System for Autonomous CTVs in Offshore Wind Farms Based on Deep Reinforcement Learning

심층 강화학습 기반 자율운항 CTV의 해상풍력발전단지 내 장애물 회피 시스템

  • Received : 2024.02.20
  • Accepted : 2024.04.08
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Crew Transfer Vessels (CTVs) are primarily used for the maintenance of offshore wind farms. Despite being manually operated by professional captains and crew, collisions with other ships and marine structures still occur. To prevent this, the introduction of autonomous navigation systems to CTVs is necessary. In this study, research on the obstacle avoidance system of the autonomous navigation system for CTVs was conducted. In particular, research on obstacle avoidance simulation for CTVs using deep reinforcement learning was carried out, taking into account the currents and wind loads in offshore wind farms. For this purpose, 3 degrees of freedom ship maneuvering modeling for CTVs considering the currents and wind loads in offshore wind farms was performed, and a simulation environment for offshore wind farms was implemented to train and test the deep reinforcement learning agent. Specifically, this study conducted research on obstacle avoidance maneuvers using MATD3 within deep reinforcement learning, and as a result, it was confirmed that the model, which underwent training over 10,000 episodes, could successfully avoid both static and moving obstacles. This confirms the conclusion that the application of the methods proposed in this study can successfully facilitate obstacle avoidance for autonomous navigation CTVs within offshore wind farms.

Keywords

References

  1. J. N. Kim, "Carbon-neutral Technological Innovation Promotion Strategy - 10 core Technology Development Directions," Korea Institute of Energy Research, 2021 (in Korean). 
  2. H. K. Cho, B. S. Park, D. H. Kang, S. S. Kim, "The Main Dactor and Counterplan for Marine Accidents in Korea," Journal of Fisheries and Marine Sciences Education, Vol. 29, No. 7, pp. 746-756, 2017 (in Korean).  https://doi.org/10.13000/JFMSE.2017.29.3.746
  3. D. G. Kim, J. S. Jeong, G. K. Park, "A Study on Ship Collision Avoidance Algorithm by COLREG," International Journal of Fuzzy Logic and Intelligent Systems, Vol. 21, No. 3, pp. 290-295, 2011 (in Korean).  https://doi.org/10.5391/JKIIS.2011.21.3.290
  4. D. G. Son, D. H. Kim, "Autonomous Navigation of Unmanned Surface Vehicle Based on Fuzzy Control Considering COLREG," Journal of Korean Institute of Intelligent Systems, Vol. 31, No. 1, pp. 11-20, 2021 (in Korean).  https://doi.org/10.5391/JKIIS.2021.31.1.011
  5. J. H. Woo, N. Kim, "Collision Avoidance for an Unmanned Surface Vehicle Using Deep Reinforcement Learning," Doctor's thesis, Seoul National University, 2018 (in Korean). 
  6. N. Wen, Y. Long, R. Zhang, G. Liu, W. Wan, D. Jiao, "COLREGs-based Path Planning for USVs Using the Deep Reinforcement Learning Strategy," Journal of Marine Science and Engineering 11, No. 12, pp. 23-34, 2023.  https://doi.org/10.3390/jmse11122334
  7. X. Xu, P. Cai, Z. Ahmed, V. S. Yellapu, W. Zhang, "Path Planning and Dynamic Collision Avoidance Algorithm Under COLREGs Via Deep Reinforcement Learning," Neurocomputing 468, pp. 181-197, 2022.  https://doi.org/10.1016/j.neucom.2021.09.071
  8. Y. J. Kim, "A Study on Optimal Design of Offshore Wind Darm Based on Array Matrix," Master's thesis, Mokpo National University, 2013 (in Korean). 
  9. J. S. Jeong, "Design of an Auto-pilot for the Towing Tank Test of Model Ship," Master's Thesis, Chungnam National University, 2005 (in Korean). 
  10. T. I. Fossen, "Guidance and Control of Ocean Vehicles," John Wiley & Sons Inc, 1994. 
  11. H. Qu, E.I. Sarda, I. R. Bertaska, K. D. Ellenrieder, "Wind Feed-forward Control of a USV," IEEE, pp. 1-10, 2015. 
  12. W. Y. Hwang, "Application of System Identification to Ship Maneuvering," PhD thesis, Massachusetts Institute of Technology, 2022. 
  13. T. P. Lillicrap, J. J. Hunt, A. Pritzel, T. Erez, Y. Tassa, D. Silver, D. Wierstra, "Continuous Control with Deep Reinforcement Learning," axXiv preprint arXiv:1509.02971, 2016. 
  14. S. Fujimoto, H. Hoof, D. Meger, "Addressing Function Approximation Error in Actor-critic Methods," PMLR, 80:1587-1596, 2018. 
  15. M. Zhan, J. Chen, C. Du, Y. Duan, "Twin Delayed Multi Agent Deep Deterministic Policy Gradient," IEEE, pp. 48-52, 2021.