Healthcare Informatics Research

Korean Society of Medical Infomatics (대한의료정보학회)
권호 표지
DOI QR코드

DOI QR Code

Augmented Reality to Localize Individual Organ in Surgical Procedure

  • Lee, Dongheon (Biomedical Research Institute, Seoul National University Hospital) ;
  • Yi, Jin Wook (Department of Surgery, Inha University Hospital) ;
  • Hong, Jeeyoung (Biomedical Research Institute, Seoul National University Hospital) ;
  • Chai, Young Jun (Department of Surgery, SMG-SNU Boramae Medical Center) ;
  • Kim, Hee Chan (Institute of Medical & Biological Engineering, Medical Research Center, Seoul National University College of Medicine) ;
  • Kong, Hyoun-Joong (Department of Biomedical Engineering, Chungnam National University Hospital, Chungnam National University College of Medicine)
  • Received : 2018.09.21
  • Accepted : 2018.10.22
  • Published : 2018.10.31
⟨ Previous Next ⟩

Abstract

Objectives: Augmented reality (AR) technology has become rapidly available and is suitable for various medical applications since it can provide effective visualization of intricate anatomical structures inside the human body. This paper describes the procedure to develop an AR app with Unity3D and Vuforia software development kit and publish it to a smartphone for the localization of critical tissues or organs that cannot be seen easily by the naked eye during surgery. Methods: In this study, Vuforia version 6.5 integrated with the Unity Editor was installed on a desktop computer and configured to develop the Android AR app for the visualization of internal organs. Three-dimensional segmented human organs were extracted from a computerized tomography file using Seg3D software, and overlaid on a target body surface through the developed app with an artificial marker. Results: To aid beginners in using the AR technology for medical applications, a 3D model of the thyroid and surrounding structures was created from a thyroid cancer patient's DICOM file, and was visualized on the neck of a medical training mannequin through the developed AR app. The individual organs, including the thyroid, trachea, carotid artery, jugular vein, and esophagus were localized by the surgeon's Android smartphone. Conclusions: Vuforia software can help even researchers, students, or surgeons who do not possess computer vision expertise to easily develop an AR app in a user-friendly manner and use it to visualize and localize critical internal organs without incision. It could allow AR technology to be extensively utilized for various medical applications.

Keywords

Acknowledgement

Supported by : Chungnam National University

References

  1. Chang H, Choi M. Big data and healthcare: building an augmented world. Healthc Inform Res 2016;22(3):153-5. https://doi.org/10.4258/hir.2016.22.3.153
  2. Bernhardt S, Nicolau SA, Soler L, Doignon C. The status of augmented reality in laparoscopic surgery as of 2016. Med Image Anal 2017;37:66-90. https://doi.org/10.1016/j.media.2017.01.007
  3. Khor WS, Baker B, Amin K, Chan A, Patel K, Wong J. Augmented and virtual reality in surgery-the digital surgical environment: applications, limitations and legal pitfalls. Ann Transl Med 2016;4(23):454. https://doi.org/10.21037/atm.2016.12.23
  4. Kim Y, Kim H, Kim YO. Virtual reality and augmented reality in plastic surgery: a review. Arch Plast Surg 2017;44(3):179-87. https://doi.org/10.5999/aps.2017.44.3.179
  5. Tang R, Ma L, Xiang C, Wang X, Li A, Liao H, et al. Augmented reality navigation in open surgery for hilar cholangiocarcinoma resection with hemihepatectomy using video-based in situ three-dimensional anatomical modeling: a case report. Medicine (Baltimore) 2017;96(37):e8083. https://doi.org/10.1097/MD.0000000000008083
  6. Soler L, Nicolau S, Pessaux P, Mutter D, Marescaux J. Real-time 3D image reconstruction guidance in liver resection surgery. Hepatobiliary Surg Nutr 2014;3(2):73-81.
  7. Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc 2016;30(10):4174-83. https://doi.org/10.1007/s00464-016-4800-6
  8. Hassan K, Dort JC, Sutherland GR, Chan S. Evaluation of software tools for segmentation of temporal bone anatomy. Stud Health Technol Inform 2016;220:130-3.

Cited by

  1. Application of a Perception Neuron ® System in Simulation-Based Surgical Training vol.8, pp.1, 2018, https://doi.org/10.3390/jcm8010124
  2. Learning media based on augmented reality (AR) increased the skill of physical examination of the integumentary system of pregnant women in midwifery students vol.35, pp.suppl2, 2018, https://doi.org/10.1016/j.gaceta.2021.10.040