Journal of Navigation and Port Research (한국항해항만학회지)

Korean Institute of Navigation and Port Research (한국항해항만학회)
권호 표지
DOI QR코드

DOI QR Code

A study of an Architecture of Digital Twin Ship with Mixed Reality

  • Received : 2022.10.05
  • Accepted : 2022.10.25
  • Published : 2022.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

As the 4th industrial revolution progresses, the application of several cutting-edge technologies such as the Internet of Things, big data, and mixed reality (MR) in relation to autonomous ships is being considered in the maritime logistics field. The aim of this study was to apply the concept of a digital twin model based on Human Machine Interaction (HMI) including a digital twin model and the role of an operator to a ship. The role of the digital twin is divided into information provision, support, decision, and implementation. The role of the operator is divided into operation, decision-making, supervision, and standby. The system constituting the ship was investigated. The digital twin system that could be applied to the ship was also investigated. The cloud-based digital twin system architecture that could apply investigated applications was divided into ship data collection (part 1), cloud system (part 2), analysis system/ application (part 3), and MR/mobile system (part 4). A Mixed Reality device HoloLens was used as an HMI equipment to perform a simulation test of a digital twin system of an 8 m battery-based electric propulsion ship.

Keywords

Acknowledgement

This research was a part of the project titled 'Development of Smart Port-Autonomous Ships Linkage Technology', funded by the Ministry of Oceans and Fisheries, Korea

References

  1. Ai, B., Jia, M., Xu, H., Xu, J., Wen, Z., Li, B. and Zhang, D.(2021), "Coverage path planning for maritime search and rescue using reinforcement learning.", Ocean Engineering, 241, 110098. https://doi.org/10.1016/J.OCEANENG.2021.110098
  2. Alexeev, V. A., Walsh, J. E., Ivanov, V. v, -, al, Dobricic, S., Russo, S., Lagunov, A. and Podorojnyak, N.(2020), "Building a Monitoring System for Bunkering Operations in the Arctic.", IOP Conference Series: Earth and Environmental Science, Vol. 459, No. 4, 042052. https://doi.org/10.1088/1755-1315/459/4/042052.
  3. Beko, M., Ivosevic, S. and Dlabac, T.(2021), "Application of Sensors in the Corrosion Monitoring of the Ship's Structural Parts." 2021 25th International Conference on Information Technology, IT 2021. https://doi.org/10.1109/IT51528.2021.9390091.
  4. Cai, D. L.(2019), "Analysis of Steering Angle Control of Ship Variable Frequency Hydraulic Steering Gear Based on Labview." Journal of Applied Science and Engineering, Vol. 22. No. 1, pp. 49-56. https://doi.org/10.6180/JASE.201903_22(1).0006.
  5. Choi, H. T., Park, J., Choi, J., Kang, M., Lee, Y., Jung, J., Kim, J., Kweon, H., Kim, J., Yoon, K. J., Kim, H. and Park, S. T.(2021), "Design and Preliminary Results of Novel Situational Awareness Systemfor Autonomous Ship based on Artificial Intelligence Techniques.", Journal of Institute of Control, Robotics and Systems, Vol. 27, No. 8, pp. 556-564. https://doi.org/10.5302/J.ICROS.2021.21.0063.
  6. Choi, J. and Yang, C. S.(2017), "Smart Escape Support System for Passenger Ship : Active Dynamic Signage & Real-time Escape Routing.", Proceedings of the Korean Institute of Navigation and Port Research Conference, pp. 79-85.
  7. Coraddu, A., Lim, S., Oneto, L., Pazouki, K., Norman, R. and Murphy, A. J.(2019), "A novelty detection approach to diagnosing hull and propeller fouling.", Ocean Engineering, Vol. 176, pp. 65-73. https://doi.org/10.1016/J.OCEANENG.2019.01.054
  8. Coraddu, A., Oneto, L., Ghio, A., Savio, S., Anguita, D. and Figari, M.(2014),. "Machine learning approaches for improving condition-based maintenance of naval propulsion plants", Proceedings of the Institution of Mechanical Engineers, Part M: Journal of Engineering for the Maritime Environment, Vol. 230, No. 1, pp. 136-153. https://doi.org/10.1177/1475090214540874.
  9. Fuller, A., Fan, Z., Day, C. and Barlow, C.(2020), "Digital Twin: Enabling Technologies, Challenges and Open Research.", IEEE Access, 8, 108952-108971. https://doi.org/10.1109/ACCESS.2020.2998358.
  10. Gaber, M., El-Banna, S. H., El-Dabah, M. and Hamad, M. S.(2021), "Intelligent Energy Management Systemfor an all-electric ship based on adaptive neuro-fuzzy inference system.", Energy Reports, 7, pp. 7989-7998. https://doi.org/10.1016/J.EGYR.2021.06.054.
  11. Guan, W., Peng, H., Zhang, X. and Sun, H.(2022), "Ship Steering Adaptive CGS Control Based on EKF Identification Method.", Journal of Marine Science and Engineering 2022, Vol. 10, No. 2, pp. 294,. https://doi.org/10.3390/JMSE10020294.
  12. IMO(2022), "MARITIME SAFETY COMMITTEE 105th session Agenda item 7 - DEVELOPMENT OF A GOAL-BASED INSTRUMENT FOR MARITIME AUTONOMOUS SURFACE SHIPS (MASS)", p.81
  13. Ji, R., Yang, Y. and Niu, J.(2021), "Ship Fire Detection Methods Based on Machine Learning." Proceedings - 2021 International Conference on Information Science, Parallel and Distributed Systems, ISPDS 2021, 231-234. https://doi.org/10.1109/ISPDS54097.2021.00052.
  14. Kawakami, T.(2016), "Communication systems for safety and security of ships.". Journal of KONES, Vol. 23, No. 1, pp. 153-160. https://doi.org/10.5604/12314005.1213547.
  15. Kritzinger, W., Karner, M., Traar, G., Henjes, J. and Sihn, W.(2018), "Digital Twin in manufacturing: A categorical literature review and classification." Vol. 51, No. 11, pp. 1016-1022. https://doi.org/10.1016/j.ifacol.2018.08.474
  16. Lakshmi, E., Priya, M. and Achari, V. S.(2021), "An overview on the treatment of ballast water in ships." Ocean & Coastal Management, 199, 105296. https://doi.org/10.1016/J.OCECOAMAN.2020.105296.
  17. Lee, E. S., Mokashi, A. J., Moon, S. Y. and Kim, G. S.(2019), "The Maturity of Automatic Identification Systems (AIS) and Its Implications for Innovation." Journal of Marine Science and Engineering 2019, Vol. 7, No. 9. pp. 287, https://doi.org/10.3390/JMSE7090287.
  18. Lee, J. H., Kong, K. J. and Jung, B. K.(2018), "Performance analysis of dynamic positioning system with loss of propulsion power of T/S NARA.", Journal of the Korean Society of Fisheries and Ocean Technology, Vol. 54, No. 2, pp. 181-187. https://doi.org/10.3796/KSFOT.2018.54.2.181.
  19. Lee, J., Kim, S., Kim, L., Kang, J., Lee, S. and Kwon, S.(2017), "A Study on Virtual Studio Application using Microsoft HoloLens." International Journal of Advanced Smart Convergence, Vol. 6, No. 4, pp. 80-87. https://doi.org/10.7236/IJASC.2017.6.4.12.
  20. Lee, J. M., Chung, W. J., Lim, D. J. and Choi, K. S.(2019), "A Study on the Development of Hall Effect Sensor for Hydraulic Locking Alarm in Ship's Steering Gear.", Journal of the Korean Society of Manufacturing Process Engineers, Vol. 18, No. 1, pp. 116-121. https://doi.org/10.14775/KSMPE.2019.18.1.116.
  21. Li, C. and Wang, D.(2018), "Knowledge-Based Engineering-based method for containership lashing bridge optimization design and structural improvement with functionally graded thickness plates.", Proceedings of the Institution of Mechanical Engineers, Part M: Journal of Engineering for the Maritime Environment. 2019, Vol. 233, No. 3, pp. 760-778.
  22. Lin, X., Wang, S., Sun, Z. and Zhang, M.(2021), "YOLO-SD: A Real-Time Crew Safety Detection and Early Warning Approach." Journal of Advanced Transportation, Vol, 2, pp. 1-12, https://doi.org/10.1155/2021/7534739.
  23. Roberts, G. W., Hancock C., Klug, F., Lienhart, W., Zuzek, N., Ligt, H.(2019), "Deflection Monitoring and frequency response of a Ship using GPS and Fibre Optic based sensors.", Joint International Symposium on Deformation Monitoring .
  24. Ma, Y., Maqsood, A., Oslebo, D. and Corzine, K.(2021), "Comparative analysis of data driven fault detection using wavelet and fourier transform for DC pulsed power load in the all-electric ship.", Conference Proceedings - IEEE Applied Power Electronics Conference and Exposition - APEC, pp. 1343-1347. https://doi.org/10.1109/APEC42165.2021.9487090.
  25. Mihai, V., Rusu, L., Alavi, A. H. and Vafai, K.(2021), "An Overview of the Ship Ventilation Systems and Measures to Avoid the Spread of Diseases." Inventions 2021, Vol. 6, No. 55. https://doi.org/10.3390/INVENTIONS6030055.
  26. Miyauchi, Y., Sawada, R., Akimoto, Y., Umeda, N. and Maki, A.(2022), "Optimization on planning of trajectory and control of autonomous berthing and unberthing for the realistic port geometry." Ocean Engineering, 245, 110390. https://doi.org/10.1016/J.OCEANENG.2021.110390.
  27. Pan, J., Hou, W., Wang, L., Lv, H., Zhang, J. and Huang, J.(2022), "Sealing state monitoring of watertight door using fiber Bragg grating sensor with arc-shaped structure.", Optical Engineering, Vol. 61, No. 4, 047102. https://doi.org/10.1117/1.OE.61.4.047102.
  28. Parsons, M. G. and Kotinis, M.(2011), "Trim and Draft Control Capability of the Variable Buoyancy Ship." Journal of Ship Production and Design, Vol. 27, No. 3, pp. 118-126. https://doi.org/10.5957/JSPD.2011.27.3.118.
  29. Perabo, F., Park, D., Zadeh, M. K., Smogeli, O. and Jamt, L.(2020), "Digital Twin Modelling of Ship Power and Propulsion Systems: Application of the Open Simulation Platform (OSP).", IEEE International Symposium on Industrial Electronics, 1265-1270. https://doi.org/10.1109/ISIE45063.2020.9152218.
  30. Poggi, L., Gaggero, T., Gaiotti, M., Ravina, E. and Rizzo, C. M.(2020), "Recent developments in remote inspections of ship structures.", International Journal of Naval Architecture and Ocean Engineering, Vol. 12, pp. 881-891. https://doi.org/10.1016/J.IJNAOE.2020.09.001.
  31. Santiago Caamano, L., Galeazzi, R., Nielsen, U. D., Miguez Gonzalez, M. and Diaz Casas, V.(2019), "Real-time detection of transverse stability changes in fishing vessels." Ocean Engineering, 189, 106369. https://doi.org/10.1016/J.OCEANENG.2019.106369.
  32. Shen, H., Liu, C., Yin, Y. and Sun, X.(2020), "Automatic Stowage of Bulk Carrier Based on Ship Floating-State Control." Journal of Ship Research, Vol. 64, No. 03, pp. 298-312. https://doi.org/10.5957/JOSR.11170070.
  33. Sheridan, T. B., Parasuraman, R.,(2005), "Human-Automation Interaction.", Reviews of Human Factors and Ergonomics. Vol. 1, pp. 89-129, https://doi.org/10.1518/155723405783703082.
  34. Simon, N., Bordoy, J., Hoflinger, F., Wendeberg, J., Schink, M., Tannhauser, R., Reindl, L. and Schindelhauer, C.(2015), "Indoor localization system for emergency responders with ultra low-power radio landmarks.", Conference Record - IEEE Instrumentation and Measurement Technology Conference, pp. 309-314. https://doi.org/10.1109/I2MTC.2015.7151285.
  35. Song, B. S., Lim, K. H., Lee, B. H., Cho, D. J., Lee, H. J., Jang, J. H. and Chang, K. H.(2019), "SMART-Navigation over Pilot LTE-Maritime: Deployment and Coexistence with PS-LTE.", IEEE Communications Magazine, Vol. 57, No. 9, pp. 126-131. https://doi.org/10.1109/MCOM.2019.1800260.
  36. Su, S., Ouyang, M., Li, T. and Wang, Q.(2021), "Application of ship electric propulsion based on internet of things system and electronic system." Microprocessors and Microsystems, 81, 103748. https://doi.org/10.1016/J.MICPRO.2020.103748.
  37. IMO(2022), "Sub-Committee on Pollution Prevention and Response (PPR 9)", p.59
  38. Tam, J. H.(2020), "Overview of performing shoreto-ship and ship-to-ship compatibility studies for LNG bunker vessels.", Journal of Marine Engineering & Technology, Vol. 21, No. 5, pp. 257-270, https://doi.org/10.1080/20464177.2020.1827489.
  39. Tao, F., Sui, F., Liu, A., Qi, Q., Zhang, M., Song, B., Guo, Z., Lu, S. C. Y. and Nee, A. Y. C.(2019), "Digital twin-driven product design framework. International Journal of Production Research", International Journal of Production Research, Vol. 57, No. 12, pp. 3935-3953. https://doi.org/10.1080/00207543.2018.1443229.
  40. Tsang, Y. P., Choy, K. L., Wu, C. H., Ho, G. T. S., Lam, H. Y. and Koo, P. S.(2017), "An IoT-based cargo monitoring system for enhancing operational effectiveness under a cold chain environment.", International Journal of Engineering Business Management, 9.
  41. Tusher, H. M., Munim, Z. H., Notteboom, T. E., Kim, T. E. and Nazir, S.(2022), "Cyber security risk assessment in autonomous shipping.", Maritime Economics & Logistics 2022, Vol. 24, No. 2, pp. 208-22,. https://doi.org/10.1057/S41278-022-00214-0.
  42. Uyanik, T., Karatug, C. and Arslanoglu, Y.(2020), "Machine learning approach to ship fuel consumption: A case of container vessel.", Transportation Research Part D: Transport and Environment, Vol. 84, 102389. https://doi.org/10.1016/J.TRD.2020.102389.
  43. Vaneeckhaute, C. and Fazli, A.(2020), "Management of ship-generated food waste and sewage on the Baltic Sea: A review.", Waste Management, Vol. 102, pp. 12-20. https://doi.org/10.1016/J.WASMAN.2019.10.030.
  44. Vassalos, D.(2020), "The role of damaged ship dynamics in addressing the risk of flooding.", Ship and Offshore Structures, Vol. 17, No. 2, pp. 279-303. https://doi.org/10.1080/17445302.2020.1827639.
  45. Wilhelm, J., Petzoldt, C., Beinke, T. and Freitag, M.(2021), "Review of Digital Twin-based Interaction in Smart Manufacturing: Enabling Cyber-Physical Systems for Human-Machine Interaction." International Journal of Computer Integrated Manufacturing, Vol. 34, No. 10, pp. 1031-1048, https://doi.org/10.1080/0951192X.2021.1963482.
  46. Xue, S., Wu, S., Tang, Q., Liu, S. and Liu, B.(2021), "Research on torsional vibration monitoring system of ship power shafting.", IOP Conference Series: Materials Science and Engineering, Vol, 1207, No. 1, pp. 012007. https://doi.org/10.1088/1757-899X/1207/1/012007.
  47. Yang, C. and Huang, F.(2016), "An overview of simulation-based hydrodynamic design of ship hull forms", Journal of Hydrodynamics, Vol. 28, No. 6, pp. 947-960. https://doi.org/10.1016/S1001-6058(16)60696-0.
  48. Yu, Y. H. and Baloch, S. R.(2019), "Real time fault detection and diagnosis system for onboard engine room.", Journal of Advanced Marine Engineering and Technology, Vol. 43, No. 10, pp. 810-815. https://doi.org/10.5916/JKOSME.2019.43.10.810.
  49. Zhang, L., Mou, J., Chen, P. and Li, M.(2021), "Path Planning for Autonomous Ships: A Hybrid Approach Based on Improved APF and Modified VO Methods.", Journal of Marine Science and Engineering 2021, Vol. 9, No. 7, pp. 761, https://doi.org/10.3390/JMSE9070761.
  50. Zhang, X., Wang, C., Jiang, L., An, L. and Yang, R.(2021), "Collision-avoidance navigation systems for Maritime Autonomous Surface Ships: A state of the art survey.", Ocean Engineering, Vol. 235, 109380. https://doi.org/10.1016/J.OCEANENG.2021.109380.
  51. Zhu, G., Guo, X., Yi, Y., Tan, W. and Ji, C.(2020), "Experiment and simulation research of evolution process for LNG leakage and diffusion.", Journal of Loss Prevention in the Process Industries, Vol. 64, 104041. https://doi.org/10.1016/J.JLP.2019.104041