Archives of Plastic Surgery

  • 제44권3호
  • /
  • Pages.179-187
  • /
  • 2017
  • /
  • 2234-6163(pISSN)
  • /
  • 2234-6171(eISSN)
대한성형외과학회 (Korean Society of Plastic and Reconstructive Surgeons)
권호 표지
DOI QR코드

DOI QR Code

Virtual Reality and Augmented Reality in Plastic Surgery: A Review

  • Kim, Youngjun (Center for Bionics, Korea Institute of Science and Technology) ;
  • Kim, Hannah (Center for Bionics, Korea Institute of Science and Technology) ;
  • Kim, Yong Oock (Department of Plastic and Reconstructive Surgery, Yonsei University College of Medicine)
  • 투고 : 2017.02.27
  • 심사 : 2017.04.21
  • 발행 : 2017.05.22
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Recently, virtual reality (VR) and augmented reality (AR) have received increasing attention, with the development of VR/AR devices such as head-mounted displays, haptic devices, and AR glasses. Medicine is considered to be one of the most effective applications of VR/AR. In this article, we describe a systematic literature review conducted to investigate the state-of-the-art VR/AR technology relevant to plastic surgery. The 35 studies that were ultimately selected were categorized into 3 representative topics: VR/AR-based preoperative planning, navigation, and training. In addition, future trends of VR/AR technology associated with plastic surgery and related fields are discussed.

키워드

참고문헌

  1. Zimmerman TG, Lanier J, Blanchard C, et al. A hand gesture interface device. In: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface; 1987 Apr 5-9; Toronto, CA. 1987. p.189-92.
  2. Chinnock C. Virtual reality in surgery and medicine. Hosp Technol Ser 1994;13:1-48.
  3. Pensieri C, Pennacchini M. Overview: virtual reality in medicine. J Virtual Worlds Res 2014;7:1-34.
  4. Geomagic. Geomagic Haptic Devices [Internet]. Rock Hill, SC: Geomagic; c2016 [cited 2017 Feb 7]. Available from: http://www.geomagic.com/en/products-landing-pages/haptic.
  5. Fushima K, Kobayashi M. Mixed-reality simulation for orthognathic surgery. Maxillofac Plast Reconstr Surg 2016;38:13. https://doi.org/10.1186/s40902-016-0059-z
  6. Tsai MD, Liu CS, Liu HY, et al. Virtual reality facial contouring surgery simulator based on CT transversal slices. Proceedings of the 5th International Conference on Bioinformatics and Biomedical Engineering; 2011 May 10-12; Wuhan, China. 2011. p.1-4.
  7. Wang Q, Chen H, Wu W, et al. Real-time mandibular angle reduction surgical simulation with haptic rendering. IEEE Trans Inf Technol Biomed 2012;16:1105-14. https://doi.org/10.1109/TITB.2012.2218114
  8. Olsson P, Nysjo F, Rodriguez-Lorenzo A, et al. Haptics-assisted virtual planning of bone, soft tissue, and vessels in fibula osteocutaneous free flaps. Plast Reconstr Surg Glob Open 2015;3:e479. https://doi.org/10.1097/GOX.0000000000000447
  9. Schendel S, Montgomery K, Sorokin A, et al. A surgical simulator for planning and performing repair of cleft lips. J Craniomaxillofac Surg 2005;33:223-8. https://doi.org/10.1016/j.jcms.2005.05.002
  10. Olsson P, Nysjo F, Hirsch JM, et al. A haptics-assisted cranio- maxillofacial surgery planning system for restoring skeletal anatomy in complex trauma cases. Int J Comput Assist Radiol Surg 2013;8:887-94. https://doi.org/10.1007/s11548-013-0827-5
  11. Zhang J, Li D, Liu Q, et al. Virtual surgical system in reduction of maxillary fracture. Proceedings of the 2015 IEEE International Conference on Digital Signal Processing (DSP); 2015 Jul 21-24; Singapore. 2015. p.1102-5.
  12. Kovler I, Joskowicz L, Weil YA, et al. Haptic computer-assisted patient-specific preoperative planning for orthopedic fractures surgery. Int J Comput Assist Radiol Surg 2015;10:1535-46. https://doi.org/10.1007/s11548-015-1162-9
  13. Cecil J, Ramanathan P, Pirela-Cruz M, et al. A Virtual Reality Based Simulation Environment for Orthopedic Surgery. In: Meersman R, Panetto H, Mishra A, et al., editors. On the Move to Meaningful Internet Systems: OTM 2014 Workshops: Confederated International Workshops: OTM Academy, OTM Industry Case Studies Program, C&TC, EI2N, INBAST, ISDE, META4eS, MSC and OnToContent; 2014 Oct 27-31; Amantea, Italy. Berlin, Heidelberg: Springer Berlin Heidelberg; 2014. p.275-85.
  14. Cecil J, Ramanathan P, Rahneshin V, et al. Collaborative virtual environments for orthopedic surgery. Proceedings of the 2013 IEEE International Conference on Automation Science and Engineering (CASE); 2013 Aug 17-20; Madison, WI. 2013. p.133-7.
  15. Shen F, Chen B, Guo Q, et al. Augmented reality patientspecific reconstruction plate design for pelvic and acetabular fracture surgery. Int J Comput Assist Radiol Surg 2013;8:169-79. https://doi.org/10.1007/s11548-012-0775-5
  16. Chan S, Li P, Locketz G, et al. High-fidelity haptic and visual rendering for patient-specific simulation of temporal bone surgery. Comput Assist Surg (Abingdon) 2016;21:85-101. https://doi.org/10.1080/24699322.2016.1189966
  17. Luciano CJ, Banerjee PP, Sorenson JM, et al. Percutaneous spinal fixation simulation with virtual reality and haptics. Neurosurgery 2013;72 Suppl 1:89-96. https://doi.org/10.1227/NEU.0b013e3182750a8d
  18. ImmersiveTouch Inc. ImmersiveTouch [Internet]. Chicago, CA: ImmersiveTouch Inc; 2017 [cited 2017 Jan 12]. Available from: http://www.immersivetouch.com.
  19. Badiali G, Ferrari V, Cutolo F, et al. Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Craniomaxillofac Surg 2014;42:1970-6. https://doi.org/10.1016/j.jcms.2014.09.001
  20. Zinser MJ, Mischkowski RA, Dreiseidler T, et al. Computerassisted orthognathic surgery: waferless maxillary positioning, versatility, and accuracy of an image-guided visualisation display. Br J Oral Maxillofac Surg 2013;51:827-33. https://doi.org/10.1016/j.bjoms.2013.06.014
  21. Mischkowski RA, Zinser MJ, Kubler AC, et al. Application of an augmented reality tool for maxillary positioning in orthognathic surgery: a feasibility study. J Craniomaxillofac Surg 2006;34:478-83. https://doi.org/10.1016/j.jcms.2006.07.862
  22. Lin L, Shi Y, Tan A, et al. Mandibular angle split osteotomy based on a novel augmented reality navigation using specialized robot-assisted arms: a feasibility study. J Craniomaxillofac Surg 2016;44:215-23. https://doi.org/10.1016/j.jcms.2015.10.024
  23. Wang J, Suenaga H, Yang L, et al. Video see-through augmented reality for oral and maxillofacial surgery. Int J Med Robot 2016;2016 Jun 9 [Epub]. http://doi.org/10.1002/rcs.1754.
  24. Choi H, Park Y, Cho H, et al. An augmented reality based simple navigation system for pelvic tumor resection. Proceedings of the 11th Asian Conference on Computer Aided Surgery (ACCAS 2015); 2015 Jul 9-11; Singapore. 2015.
  25. Choi H, Cho B, Masamune K, et al. An effective visualization technique for depth perception in augmented realitybased surgical navigation. Int J Med Robot 2016;12:62-72. https://doi.org/10.1002/rcs.1657
  26. Wu F, Chen X, Lin Y, et al. A virtual training system for maxillofacial surgery using advanced haptic feedback and immersive workbench. Int J Med Robot 2014;10:78-87. https://doi.org/10.1002/rcs.1514
  27. Lin Y, Wang X, Wu F, et al. Development and validation of a surgical training simulator with haptic feedback for learning bone-sawing skill. J Biomed Inform 2014;48:122-9. https://doi.org/10.1016/j.jbi.2013.12.010
  28. Seah TE, Barrow A, Baskaradas A, et al. A virtual reality system to train image guided placement of kirschner-wires for distal radius fractures. In: Bello F, Cotin S, editors. Biomedical Simulation: 6th International Symposium, ISBMS 2014, 2014 Oct 16-17; Strasbourg, FR. Cham: Springer International Publishing; 2014. p.20-9.
  29. Thomas GW, Johns BD, Kho JY, et al. The validity and reliability of a hybrid reality simulator for wire navigation in orthopedic surgery. IEEE Trans Hum Mach Syst 2015;45:119-25. https://doi.org/10.1109/THMS.2014.2339324
  30. Swemac. TraumaVision-medical orthopedic simulator [Internet]. Linkoping, SE: Swemac; c2017 [cited 2017 Jan 12]. Available from: http://www.swemac.com/simulators/traumavision.
  31. OssoVR. OssoVR [Internet]. San Francisco, CA: OssoVR [cited 2017 Feb 6]. Available from: http://ossovr.com.
  32. Vankipuram M, Kahol K, McLaren A, et al. A virtual reality simulator for orthopedic basic skills: a design and validation study. J Biomed Inform 2010;43:661-8. https://doi.org/10.1016/j.jbi.2010.05.016
  33. Wong D, Unger B, Kraut J, et al. Comparison of cadaveric and isomorphic virtual haptic simulation in temporal bone training. J Otolaryngol Head Neck Surg 2014;43:31.
  34. Qiong W, Hui C, Wen W, et al. Impulse-based rendering methods for haptic simulation of bone-burring. IEEE Trans Haptics 2012;5:344-55. https://doi.org/10.1109/TOH.2011.69
  35. Lemole GM Jr, Banerjee PP, Luciano C, et al. Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery 2007;61:142-8. https://doi.org/10.1227/01.neu.0000279734.22931.21
  36. Alaraj A, Luciano CJ, Bailey DP, et al. Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback. Neurosurgery 2015;11 Suppl 2:52-8.
  37. Sutherland C, Hashtrudi-Zaad K, Sellens R, et al. An augmented reality haptic training simulator for spinal needle procedures. IEEE Trans Biomed Eng 2013;60:3009-18. https://doi.org/10.1109/TBME.2012.2236091
  38. Tsai MD, Hsieh MS. Computer based system for simulating spine surgery. Proceedings of the 22nd IEEE International Symposium on Computer-Based Medical Systems; 2009 Aug 2-5; Albuquerque, NM. 2009. p.1-8.
  39. CAE Healthcare Inc. NeuroVR [Internet]. Montreal, CA: Cae Healthcare; 2017 [cited 2017 Jan 12]. Available from: http://caehealthcare.com/eng/interventional-simulators/neurovr.
  40. Woo T, Kraeima J, Kim YO, et al. Mandible reconstruction with 3D virtual planning. J Int Soc Simul Surg 2015;2:90-3.
  41. Burdea G, Coiffet P. Virtual reality technology. Hoboken, NJ: J. Wiley-Interscience; 2003.
  42. Hackett M, Proctor M. Three-dimensional display technologies for anatomical education: a literature Review. J Sci Educ Technol 2016;25:641-54. https://doi.org/10.1007/s10956-016-9619-3
  43. Makino Y, Furuyama Y, Inoue S, et al. HaptoClone (Haptic-Optical Clone) for mutual tele-environment by real-time 3D image transfer with midair force Feedback. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems; 2016; Santa Clara, CA: ACM; 2016. p.1980-90.

피인용 문헌

  1. Discussion: Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room vol.140, pp.5, 2017, https://doi.org/10.1097/prs.0000000000003817
  2. Direct augmented reality computed tomographic angiography technique (ARC): an innovation in preoperative imaging vol.41, pp.4, 2017, https://doi.org/10.1007/s00238-018-1395-2
  3. Augmented reality in dentistry: a current perspective vol.76, pp.7, 2017, https://doi.org/10.1080/00016357.2018.1441437
  4. Role of virtual reality in congenital heart disease vol.13, pp.3, 2018, https://doi.org/10.1111/chd.12587
  5. Delayed reconstruction of posttraumatic facial deformities vol.61, pp.12, 2018, https://doi.org/10.5124/jkma.2018.61.12.740
  6. Effects of mixed reality head-mounted glasses during 90 minutes of mental and manual tasks on cognitive and physiological functions vol.6, pp.None, 2018, https://doi.org/10.7717/peerj.5847
  7. A low‐cost multimodal head‐mounted display system for neuroendoscopic surgery vol.8, pp.1, 2018, https://doi.org/10.1002/brb3.891
  8. Virtual Reality in Endoscopic Sinus Surgery and Facial Plastic & Reconstructive Surgery vol.61, pp.5, 2017, https://doi.org/10.3342/kjorl-hns.2018.00283
  9. Augmented reality and dynamic infrared thermography for perforator mapping in the anterolateral thigh vol.45, pp.3, 2017, https://doi.org/10.5999/aps.2017.01375
  10. Augmented Reality to Localize Individual Organ in Surgical Procedure vol.24, pp.4, 2018, https://doi.org/10.4258/hir.2018.24.4.394
  11. Manufacturing Better Outcomes in Cardiovascular Intervention: 3D Printing in Clinical Practice Today vol.20, pp.12, 2017, https://doi.org/10.1007/s11936-018-0692-1
  12. Applications of Augmented Reality in Otolaryngology: A Systematic Review vol.159, pp.6, 2018, https://doi.org/10.1177/0194599818796476
  13. Comparison of Two Innovative Strategies Using Augmented Reality for Communication in Aesthetic Dentistry: A Pilot Study vol.2019, pp.None, 2017, https://doi.org/10.1155/2019/7019046
  14. Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography vol.14, pp.3, 2017, https://doi.org/10.1007/s11548-018-1814-7
  15. Augmented Reality in Emergency Medicine: A Scoping Review vol.21, pp.4, 2017, https://doi.org/10.2196/12368
  16. Feasibility of Virtual Reality Combined with Isocentric Navigation in Transforaminal Percutaneous Endoscopic Discectomy: A Cadaver Study vol.11, pp.3, 2019, https://doi.org/10.1111/os.12473
  17. Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care vol.291, pp.3, 2017, https://doi.org/10.1148/radiol.2019182210
  18. Standard Terminology System Referenced by 3D Human Body Model vol.17, pp.2, 2017, https://doi.org/10.6109/jicce.2019.17.2.91
  19. Acceptability and effectiveness of a 3D virtual reality cardiovascular lesson vol.9, pp.3, 2017, https://doi.org/10.12968/ippr.2019.9.3.67
  20. The application of virtual reality and augmented reality in Oral & Maxillofacial Surgery vol.19, pp.1, 2019, https://doi.org/10.1186/s12903-019-0937-8
  21. Are augmented reality headsets in surgery a dead end? vol.16, pp.12, 2017, https://doi.org/10.1080/17434440.2019.1693891
  22. Augmented Reality Technology-based Dental Radiography Simulator for Preclinical Training and Education on Dental Anatomy vol.17, pp.4, 2019, https://doi.org/10.6109/jicce.2019.17.4.274
  23. La réalité virtuelle au service de l’apprentissage de la géométrie descriptive : retour vers Monge vol.82, pp.None, 2020, https://doi.org/10.1051/shsconf/20208204001
  24. Application of augmented reality to surgical practice: A pilot study using the ODG R7 Smartglasses vol.45, pp.1, 2017, https://doi.org/10.1111/coa.13460
  25. Supporting fibula free flap harvest with augmented reality: A proof‐of‐concept study vol.130, pp.5, 2017, https://doi.org/10.1002/lary.28090
  26. Edge caching and computing in 5G for mobile augmented reality and haptic internet vol.158, pp.None, 2017, https://doi.org/10.1016/j.comcom.2020.04.054
  27. The evolution of virtual reality in shoulder and elbow surgery vol.4, pp.2, 2017, https://doi.org/10.1016/j.jseint.2020.02.005
  28. Representation of Chiseling Operation Using Force Display with Two Degree-of-freedom Admittance Control vol.56, pp.6, 2017, https://doi.org/10.9746/sicetr.56.333
  29. Organic Haptics: Intersection of Materials Chemistry and Tactile Perception vol.30, pp.29, 2017, https://doi.org/10.1002/adfm.201906850
  30. Effect of virtual reality-simulated exercise on sympathovagal balance vol.15, pp.7, 2017, https://doi.org/10.1371/journal.pone.0235792
  31. Use of Simulation in Plastic Surgery Training vol.8, pp.7, 2017, https://doi.org/10.1097/gox.0000000000002896
  32. Visuo-Haptic Mixed Reality Simulation Using Unbound Handheld Tools vol.10, pp.15, 2017, https://doi.org/10.3390/app10155344
  33. Head‐Mounted Devices for Noninvasive Cancer Imaging and Intraoperative Image‐Guided Surgery vol.30, pp.37, 2017, https://doi.org/10.1002/adfm.202000185
  34. Role of Breast MRI in the Evaluation and Detection of DCIS: Opportunities and Challenges vol.52, pp.3, 2017, https://doi.org/10.1002/jmri.26985
  35. Virtual Reality Simulation and Augmented Reality-Guided Surgery for Total Maxillectomy: A Case Report vol.10, pp.18, 2017, https://doi.org/10.3390/app10186288
  36. Using Virtual Reality for Movement System Examination in a Doctor of Physical Therapy Curriculum vol.35, pp.4, 2017, https://doi.org/10.1097/jte.0000000000000204
  37. Lessons Learned: Medical Library Pilot Testing of a Virtual Reality Anatomy Lab vol.18, pp.1, 2017, https://doi.org/10.1080/15424065.2021.1882920
  38. Plastic Surgery Fellowship at Nippon Medical School Hospital: An Integrative Approach to Modern Plastic Surgery Education vol.9, pp.1, 2021, https://doi.org/10.1097/gox.0000000000003367
  39. BlocklyXR: An Interactive Extended Reality Toolkit for Digital Storytelling vol.11, pp.3, 2017, https://doi.org/10.3390/app11031073
  40. Virtual Reality in Preoperative Planning of Adolescent Idiopathic Scoliosis Surgery Using Google Cardboard vol.18, pp.1, 2017, https://doi.org/10.14245/ns.2040574.287
  41. The role of simulation in training breast surgeons: a systematic review vol.103, pp.5, 2021, https://doi.org/10.1308/rcsann.2020.7138
  42. Virtual Reality (VR) Simulation and Augmented Reality (AR) Navigation in Orthognathic Surgery: A Case Report vol.11, pp.12, 2021, https://doi.org/10.3390/app11125673
  43. Virtual Reality vs. Tablet Video as an Experiential Education Platform for Pediatric Patients Undergoing Chest Radiography: A Randomized Clinical Trial vol.10, pp.11, 2017, https://doi.org/10.3390/jcm10112486
  44. The utility of augmented reality in lateral skull base surgery: A preliminary report vol.42, pp.4, 2021, https://doi.org/10.1016/j.amjoto.2021.102942
  45. Multi-Shape Free-Form Deformation Framework for Efficient Data Transmission in AR-Based Medical Training Simulators vol.11, pp.21, 2017, https://doi.org/10.3390/app11219925
  46. The Use of Live Streaming Technologies in Surgery : A Review of the Literature vol.88, pp.1, 2017, https://doi.org/10.1097/sap.0000000000002909
  47. Augmented, Virtual and Mixed Reality in Dentistry: A Narrative Review on the Existing Platforms and Future Challenges vol.12, pp.2, 2017, https://doi.org/10.3390/app12020877
  48. An Augmented Reality Platform with Hand Gestures-Based Navigation for Applications in Image-Guided Surgery: Prospective Concept Evaluation by Surgeons vol.38, pp.2, 2017, https://doi.org/10.1080/10447318.2021.1926116