KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Augmented-Reality Survey: from Concept to Application

  • Kim, Soo Kyun (Department of Game Engineering, Paichai University) ;
  • Kang, Shin-Jin (School of Games, Hongik University) ;
  • Choi, Yoo-Joo (Department of New Media Content, Seoul Media Institute of Technology) ;
  • Choi, Min-Hyung (Department of Computer Science and Engineering, University of Colorado Denver) ;
  • Hong, Min (Department of Computer Software Engineering, Soonchunhyang University)
  • Received : 2016.11.07
  • Accepted : 2017.01.16
  • Published : 2017.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

The recent advances in the field of augmented reality (AR) have shown that the technology is a fundamental part of modern immersive interactive systems for the achievement of user engagement and a dynamic user experience. This survey paper presents the descriptions of a variety of the new AR explorations, and the issues that are relevant to the contemporary development of the fundamental technologies and applications are discussed. Most of the literature regarding the pertinent topics-taxonomy, the core tracking and sensing technologies, the hardware and software platforms, and the domain-specific applications-are then chronologically surveyed, and in varying detail, this is supplemented with the cited papers. This paper portrays the diversity of the research regarding the AR field together with an overview of the benefits and the limitations of the competing and complementary technologies.

Keywords

References

  1. I. Sutherland, "A head-mounted three dimensional display," in Proc. of the Fall Joint Computer Conference, pp. 757-764, Dec. 9-11, 1968.
  2. google glasses
  3. Gear VR
  4. Ihsan Rabbi and Sehat Ullah, "A Survey on Augmented Reality Challenges and Tracking," Acta Graphica ,vol. 24, no 1-2, 2013.
  5. Feng Zhou, Henry Been-Lirn Duh and Mark Billinghurst, "Trends in augmented reality tracking, interaction and display: A Review of Ten Years of ISMAR," in Proc. of ISMAR '08: Proc. 7th International Symposium on Mixed and Augmented Reality, pp. 193-202, September, 2008.
  6. Ronald T. Azuma, "A survey of augmented reality," Presence: Teleoperators and Virtual Environments, vol. 6, no.4, pp.355-385, August, 1997. https://doi.org/10.1162/pres.1997.6.4.355
  7. D.W.F. van Krevelen and R. Poelman, "A survey of augmented reality technologies, applications and limitations," The International Journal of Virtual Reality, vol 9, no.2, pp.1-20, 2010.
  8. Julie Carmigniani, Borko Furht, Marco Anisetti, Paolo Ceravolo, Ernesto Damiani and Misa Ivkovic, "Augmented reality technologies, systems and application," Multimedia Tools and Applications, vol. 51, no. 1, pp. 341-377, January, 2011. https://doi.org/10.1007/s11042-010-0660-6
  9. X. Wang, S. K. Ong and A. Y. C. Nee, "A comprehensive survey of augmented reality assembly research," Advances in Manufacturing, vol. 4, no. 1, pp. 1-22, March, 2016. https://doi.org/10.1007/s40436-015-0131-4
  10. Federico Manuri and Andrea Sanna, "A survey on applications of augmented reality," Advances in Computer Science: an International Journal, vol. 5, issue 1, no. 19, pp. 18-27, January, 2016.
  11. R. Azuma and G. Bishop, "Improving static and dynamic registration in a see-through HMD," Computer Grphics (Proceedings of SIGGRAPH '94), pp. 197-204, 1994.
  12. M. Bajura and U. Neumann, "Dynamic registration correction in video-based augmented reality systems," IEEE Computer Graphics and Applications, vol. 15, no. 5, pp. 52-60, 1995. https://doi.org/10.1109/38.403828
  13. J. P. Mellor, "Enhanced reality visualization in a surgical environment," Master's thesis, Dept.of Electrical Enginerring, MIT, 1995.
  14. J. P. Mellor, "Realtime camera calibration for enhanced reality visualization," in Proc. of Computer Vision, Virtual Reality, and Robotics in Medicine '95, pp. 471-475, 1995.
  15. U. Neumann and Y. Cho, "A self-tracking augmented reality system," in Proc. of Virtual Reality Software and Technology '96, pp. 109-115, 1996.
  16. A. State, G. Hirota, D. T. Chen, B. Garrett and M. Livingston, "Superior augmented reality registration by integrating landmark tracking and magnetic tracking," Computer Graphics (Proceedings of SIGGRAPH '96), pp. 429-438, 1996.
  17. T.H. Hollerer and S.K. Feiner, "Mobile Augmented Reality," Telegeoinformatics: Location-based Computing and Services, CRC Press, Mar. 2004. ISBN 0-4153-6976-2.
  18. Markus Aleksy and Elina Vartiainen, "Visualization Approaches in Augmented Reality Applications," Volume 9228 of the series Lecture Notes in Computer Science, pp. 84-90, August 2015.
  19. Mobile augmented reality
  20. Y.K. Chang, U.K. Woo, D.C. Kim and C.S. Shin, "Mobile Augmented Reality Technology Trend," OSIA Standards & Technology Review, vol.38, no.1, 2010.
  21. Thomas P. Caudell and David W. Mizell, "Augmented reality: An application of heads-up display technology to manual manufacturing processes," System Sciences, in Proc. of the Twenty-Fifth Hawaii International Conference on, vol. 2, pp. 659-669, 1992.
  22. Optical Tracking Explained
  23. Positional tracking
  24. Yuko Uematsu and Hideo Saito, "Vision-based Augmented Reality Applications," Computer vision I-Tech, Vienna, Austria, 2008.
  25. U. Neumann and A. Majoros, "Cognitive, performance and systems issues for augmented reality applications in manufacturing and maintenance," IEEE Virtual Reality Annual International Symposium, pp. 4-11. 1998.
  26. D. H. Shin and P.S. Dunston, "Identification of application areas for augmented reality in industrial construction based on technology suitability," Automation in Construction, vol. 17, pp. 882-894, 2008. https://doi.org/10.1016/j.autcon.2008.02.012
  27. Akash Bapat, Enrique Dunn and Jan-michael Frahm, "Kilo-Hertz 6-DoF visual tracking using an egocentric cluster of rolling shutter cameras," in Proc. of Mixed and Augmented Reality(ISMAR), 15th IEEE International Symposium on, 2016
  28. C. Harris and M. Stephens, "A combined corner and edge detector," in Proc. of Fourth Alvey Vision Conference, pp. 147-151, 1988.
  29. S. M. Smith and J. M. Brady, "SUSAN - a new approach to low-level image processing," International Journal of Computer Vision, vol. 23, no. 1, pp. 45-78, 1997. https://doi.org/10.1023/A:1007963824710
  30. E. Rosten, "Machine Learning for High-speed Corner Detection," in Proc. of 9th European Conference on Computer Vision, pp. 430-443, May.2006.
  31. G. Hua and A. Akbarzadeh, "A robust elastic and partial matching metric for face recognition," in Proc. of 2009 IEEE 12th International Conference on Computer Vision, pp. 2082-2089, September , 2009.
  32. D.G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, no. 2, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  33. H. Bay, T. Tuytelaars and L. Van Gool, "Surf: Speeded up robust features," in Proc. of European Conference on Computer Vision, 2006.
  34. Michael Calonder, Vincent Lepetit, Christoph Strecha, and Pascal Fua, "BRIEF: Binary Robust Independent Elementary Features," in Proc. of 11th European Conference on Computer Vision (ECCV), Heraklion, Crete. LNCS Springer, September ,2010.
  35. Ethan Rublee, Vincent Rabaud, Kurt Konolige and Gary R. Bradski, "ORB: An efficient alternative to SIFT or SURF," in Proc. of 2011 International conference on computer vision, IEEE, pp. 2564-2571, 2011.
  36. M. Ozuysal, P. Fua and V. Lepetit, "Fast Keypoint Recognition in Ten Lines of Code," in Proc. of IEEE Conference of Computer Vision and Pattern Recognition, July.2007.
  37. B. Lee and J. Chun, "Interactive manipulation of augmented objects in marker-less AR using vision based hand interaction," in Proc. of 7th International Conference on Information Technology, pp. 398-403, 2010.
  38. Tan W, Liu U and Dong Z, "Robust monocular SLAM in dynamic environments," in Proc. of the 12th IEEE international symposium on mixed and augmented reality (ISMAR 2013), Adelaide, pp. 209-218, 2013.
  39. Petit A, Marchand E and Kanani K, "Augmenting markerless complex 3D objects by combining geometrical and color edge information," in Proc. of the 12th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Adelaide, pp. 287-288, 2013.
  40. Eric Marchand, Hideaki Uchiyama and Fabien Spindler, "Pose estimation for augmented reality: a hands-on survey," IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers, 2016.
  41. C.E. Hughes, C. B. Stapleton, D.E. Hughes and E. M. Smith, "Mixed reality in education, entertainment, and training," IEEE Computer Graphics and Applications, pp. 24-30, 2005.
  42. G. Schall, A. Mulloni and G. Reitmayr, "North-centred orientation tracking on mobile phones," Mixed and Augmented Reality(ISMAR), in Proc. of the 9th IEEE International Symposium, 2010.
  43. G. Bleser, G. Hendeby and M. Miezal, "Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances," in Proc. of Mixed and Augmented Reality(ISMAR), 10th IEEE International Symposium, 2011.
  44. Augmented reality
  45. J.S. Bang and U.J. Choi, "The Trend of Technology and Prospect of Augmented Reality," Emerging Issue Report, KISTI, 2008.
  46. Magic make-up mirror
  47. Pereira, Pedro, Nuno Matos, and Aderito Marcos, "ARK: augmented reality kiosk," Human-Computer Interaction-Theory and Practice(PART II), Edited by Constantine Stephanidis, Julie Jacko(Vol. 2 of the Proceedings of HCI International 2003, 22-27 June, Crete, Greece) pp.168-172, 2003. ISBN: 0-8058-4931-9
  48. S. E. Reed, O. Kreylos, S. Hsi, L. H. Kellogg, G. Schladow, M. B. Yikilmaz, H. Segale, J. Silverman, S. Yalowitz and E. Sato, "Shaping watersheds exhibit: An interactive, augmented reality sandbox for advancing earth science education," in Proc. of AGU Fall Meeting Abstracts ,vol. 1, pp. 1, December, 2014.
  49. Mauro Figueiredo, Maria-Angeles Cifredo-Chacon and Vitor Goncalves, "Learning Programming and Electronics with Augmented Reality," Universal Access in Human-Computer Interaction. Users and Context Diversity, Volume 9739 of the series Lecture Notes in Computer Science, pp 57-64, 2016.
  50. James Swanson, "MFS Launches Fund," Fund Action, 2014. ISSN: 1054-5956.
  51. Aiteanu, Dorin, Bernd Hillers, and A. Graser, "A step forward in manual welding: demonstration of augmented reality helmet," Mixed and Augmented Reality, in Proc. of The Second IEEE and ACM International Symposium on, IEEE, 2003.
  52. Microsoft HoloLens
  53. Z. Lv, L. Feng, H. Li, and S. Feng, "Hand-free motion interaction on google glass," SIGGRAPH Asia 2014 Mobile Graphics and Interactive Application, ACM, pp.21, 2014.
  54. Vuzix Augmented product
  55. Meta 2 Development Kit
  56. LAFORGE Shima
  57. ATHEER AiR$^{TM}$
  58. IEEE Spectrum Augmented Reality in a Contact Lens
  59. D. Stredney and S. J. Weghorst, "The virtual retinal display: a new technology for virtual reality and augmented vision in medicine," Medicine Meets Virtual Reality: Art, Science, Technology: Healthcare (R) Evolution, vol. 50, pp.252, 1998. ISSN 0926-9630
  60. Chryssi Birliraki, George Margetis , Nikolaos Patsiouras, Giannis Drossis and Constantine Stephanidis, "Enhancing the Customers' Experience Using an Augmented Reality Mirror," in Proc. of HCI International 2016 - Posters' Extended Abstracts, Volume 618 of the series Communications in Computer and Information Science , pp 479-484, 2016.
  61. VR Interfaces Phantom
  62. Abdulmotaleb El Saddik, Mauricio Orozco, Mohamad Eid and Jongeun Cha, "Multimedia Haptics," Haptics Technologies, Part of the series Springer Series on Touch and Haptic Systems pp 145-182, 2011.
  63. Maestro AR$^{TM}$
  64. Dhiraj Amin and Sharvari Govilkar, "Comparative study of augmented reality SDKs," International Journal on Computational Science & Applications (IJCSA), vol.5, no.1, 2015.
  65. Steve Bourgeois , Boris Meden , Vincent Gay-Bellile , Mohamed Tamaazousti and Sebastian Knodel, "Modeling, Tracking, Annotating and Augmenting a 3D Object in less than 5 Minutes," Vision and Content Engineering Laboratory, Point Courrier 94, Gif-sur-Yvette, F-91191 France, 2016.
  66. Sylvain Cardin, Howard Ogden, Daniel Perez-Marcos, John Williams and Tomo Ohno, "Neurogoggles for Multimodal Augmented Reality," in Proc. of the 7th Augmented Human International Conference 2016, no. 48, 2016.
  67. Jinwook, ShimYoonsik, Yang Nahyung, Kang Jonghoon, SeoTack-Don Han, "Gesture-based interactive augmented reality content authoring system using HMD," Virtual Reality, vol 20, Issue 1, pp 57-69, 2016. https://doi.org/10.1007/s10055-016-0282-z
  68. Giles Westerfield, Antonija Mitrovic and Mark Billinghurst, "Intelligent Augmented Reality Training for Motherboard Assembly," International Journal of Artificial Intelligence in Education, vol. 25, Issue 1, pp. 157-172, 2015. https://doi.org/10.1007/s40593-014-0032-x
  69. Zhihan Lv, Alaa Halawani, Shengzhong Feng, Shafiq ur Rehman and Haibo Li, "Touch-less interactive augmented reality game on vision-based wearable device," Personal and Ubiquitous Computing , vol. 19, Issue 3, pp. 551-567, 2015. https://doi.org/10.1007/s00779-015-0844-1
  70. Joaquin ,Cubillo Sergio Martin, Manuel Castro and Ivica Boticki, "Preparing augmented reality learning content should be easy: UNED ARLE-an authoring tool for augmented reality learning environments," Computer Applications in Engineering Education, vol. 23, Issue 5, pp. 778-789, 2015. https://doi.org/10.1002/cae.21650
  71. Anuruddha Hettiarachchi and Daniel Wigdor, "Annexing Reality: Enabling Opportunistic Use of Everyday Objects as Tangible Proxies in Augmented Reality," in Proc. of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 1957-1967, 2016.
  72. Robin De Croon, Joris Klerkx and Erik Duval, "A Proof-of-Concept Visualization to Increase Comprehension of Personal Medication Schemes," in Proc. of 2016 10th International Conference On Pervasive Computing Technologies For Healthcare (PervasiveHealth), pp.1-8, 2016.
  73. Julio Cabero and Julio Barroso, "Posibilidades educativas de la Realidad Aumentada," NEW APPROACHES IN EDUCATIONAL RESEARCH, vol. 5, no. 1, pp. 46-52, 2016.
  74. Franceli L. Cibrian, Deysi H. Ortega, Lizbeth Escobedo and Monica Tentori, "Reflections from a Long-term Deployment Study to Design Novel Interactive Surfaces for Children with Autism," Ambient Intelligence for Health, Volume 9456 of the series Lecture Notes in Computer Science, pp. 167-176, 2015.
  75. Ohan Oda, Carmine Elvezio, Mengu Sukan, Steven Feiner and Barbara Tversky, "Virtual Replicas for Remote Assistance in Virtual and Augmented Reality," in Proc. of UIST '15 Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, pp. 405-415, 2015.
  76. Marianna Kopsida, Ioannis Brilakis and Patricio Antonio Vela, "A Review of Automated Construction Progress Monitoring and Inspection Methods," in Proc. of the 32nd CIB W78 Conference 2015, pp.421-431, 2015.
  77. Kaikai Liu and Xiaolin Li, "Enabling Context-Aware Indoor Augmented Reality via Smartphone Sensing and Vision Tracking," ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), vol. 12, Issue 1, no. 15, 2015.
  78. George Koutromanos, Alivisos Sofos and Lucy Avraamidou, "The use of augmented reality games in education: a review of the literature," Educational Media International, vol. 52, Issue 4, pp. 253-271, 2015. https://doi.org/10.1080/09523987.2015.1125988
  79. Gravity jack
  80. Vlasios Kasapakis , Damianos Gavalas, "Determining Field of View in Outdoors Augmented Reality Applications," Ambient Intelligence, Volume 9425 of the series Lecture Notes in Computer Science, pp 344-348, 2015.
  81. Gheric Speiginer, Blair MacIntyre , Jay Bolter, Hafez Rouzati, Amy Lambeth, Laura Levy, Laurie Baird, Maribeth Gandy, Matt Sanders and 4 more, "The Evolution of the Argon Web Framework Through Its Use Creating Cultural Heritage and Community-Based Augmented Reality Applications," Human-Computer Interaction: Users and Contexts, Volume 9171 of the series Lecture Notes in Computer Science, pp.112-124, 2015.
  82. Lamees Mahmoud Mohd Said Al Qassem, Hessa Al Hawai, Shayma Al Shehhi, M. Jamal Zemerly and Jason W. P. Ng, "AIR- EDUTHCH: Augmented immersive reality (AIR) technology for high school Chemistry education," in Proc. of 2016 IEEE Global Engineering Education Conference (EDUCON), DOI: 10.1109/EDUCON.2016.7474650, Electronic ISSN: 2165-9567, 2016.
  83. STAR 1200XLD
  84. Microsoft HoloLens
  85. Google Magic Leap
  86. Feiner, Steven and MacIntyre, Blair and Seligmann, Doree, "Annotating the real world with knowledge-based graphics on a see-through head-mounted display," in Proc. of the conference on Graphics interface'92, pp. 78-85, 1992.
  87. R. T. Whitaker, C. Crampton, D. E. Breen, M. Tuceryan, and E. Rose, "Object calibration for augmented reality," Computer Graphics Forum, Wiley Online Library, pp. 15-27, 1995.
  88. Hyundai Virtual Guide
  89. Hannes Kaufmann and Dieter Schmalstieg, "Mathematics and geometry education with collaborative augmented reality," Computers & Graphics, vol. 27, no. 3, pp. 339-345, June, 2003. https://doi.org/10.1016/S0097-8493(03)00028-1
  90. AR Flashcards
  91. Anatomy 4D
  92. Elements 4D
  93. Arloon Geometry
  94. SkyView
  95. R. Cavallaro, "The FoxTrax Hockey Puck Tracking System," IEEE Computer Graphics and Applications, vol. 17, no. 2, pp. 6-12, April, 1997. https://doi.org/10.1109/38.574652
  96. R. Azuma, Y. Balliot, R. Behringer, S. Feiner, S. Julier and B. MacIntyre, "Recent Advances in Augmented Reality," Computers & Graphics, vol. 21, no. 6, pp. 34-47, November, 2001.
  97. J. Pair, J. Wilson, J. Chastine and M. Gandy, "The Duran Duran Project: The Augmented Reality Toolkit in Live Performance," The First IEEE International Augmented Reality Toolkit Workshop, vol.1, 2002.
  98. Pokemon GO
  99. EZface
  100. Starbucks Cup Magic
  101. Field Trip
  102. Varia Vision
  103. Metro AR Pro
  104. Yelp Monocle
  105. Google Translate
  106. T. Hakky, R.M. Daniel, I. Larry, E.S. Philippe, E. C. Rafael, "Augmented Reality Assisted Urologic Surgery (ARAUS): A surgical training tool," The Journal of Urology, vol. 193, no. 4, pp.e271, May, 2015.
  107. Peters and Terry, "Augmented-Reality Assisted Surgery on the Beating Heart," IEEE Toronto, 2015.
  108. Virtual and augmented reality in surgical training: an interview with Dr Shafi Ahmed
  109. AccuVein Vein Viewing System
  110. VA-ST Smart Specs
  111. Saagara AR
  112. L. Gun, "CityViewAR outdoor AR visualization," in Proc. of the 13th International Conference of the NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction, pp. 97, 2012.
  113. Bentley Systems
  114. JBKnowledge Smart Reality
  115. BIMevoke 3D Guided Maintenance application

Cited by

  1. UTAUT2를 응용한 4차 산업 기술수용에 관한 연구: 증강현실(AR)과 드론 서비스를 중심으로 vol.26, pp.6, 2017, https://doi.org/10.21219/jitam.2019.26.6.029
  2. Comparative study of AR versus video tutorials for minor maintenance operations vol.79, pp.11, 2017, https://doi.org/10.1007/s11042-019-08437-9
  3. Real-time Human Pose Estimation using RGB-D images and Deep Learning vol.21, pp.3, 2020, https://doi.org/10.7472/jksii.2020.21.3.113
  4. Empirical Comparison of Deep Learning Networks on Backbone Method of Human Pose Estimation vol.21, pp.5, 2017, https://doi.org/10.7472/jksii.2020.21.5.21