International Journal of Internet, Broadcasting and Communication

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Implementation of AR Remote Rendering Techniques for Real-time Volumetric 3D Video

  • Lee, Daehyeon (Graduate School of Smart Convergence, Kwangwoon University) ;
  • Lee, Munyong (Graduate School of Smart Convergence, Kwangwoon University) ;
  • Lee, Sang-ha (Dept. of Electronic Engineering, Kwangwoon University) ;
  • Lee, Jaehyun (Dept. Plasma Bio Display, Kwangwoon University) ;
  • Kwon, Soonchul (Graduate School of Smart Convergence, Kwangwoon University)
  • Received : 2020.03.17
  • Accepted : 2020.03.29
  • Published : 2020.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, with the growth of mixed reality industrial infrastructure, relevant convergence research has been proposed. For real-time mixed reality services such as remote video conferencing, the research on real-time acquisition-process-transfer methods is required. This paper aims to implement an AR remote rendering method of volumetric 3D video data. We have proposed and implemented two modules; one, the parsing module of the volumetric 3D video to a game engine, and two, the server rendering module. The result of the experiment showed that the volumetric 3D video sequence data of about 15 MB was compressed by 6-7%. The remote module was streamed at 27 fps at a 1200 by 1200 resolution. The results of this paper are expected to be applied to an AR cloud service.

Keywords

References

  1. Luis Munoz-Saavedra, Lourdes Miro-Amarante, and Manuel Dominguez-Morales, "Augmented and Virtual Reality Evolution and Future Tendency," Applied Sciences, Vol. 10, No. 1, 2020. DOI: http://doi.org/10.3390/app10010322.
  2. Kiara Ottogalli, Daniel Rosquete, Aiert Amundarain, Iker Aguinaga, and Diego Borro, "Flexible Framework to Model Industry 4.0 Processes for Virtual Simulators," Applied Sciences, Vol. 9, No. 23, 2019. DOI: http://doi.org/10.3390/app9234983.
  3. Shu Shi, Klara Nahrstedt, and Roy Campbell, "A Real-Time Remote Rendering System for Interactive Mobile Graphics," ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 8, No. 3s, Article 46, 2012. DOI: http://doi.org/10.1145/2348816.2348825.
  4. Lee, K., Chu, D., Cuervo, E., Kopf, J., Degtyarev, Y.,Grizan, S., Wolman, A., and Flinn, J. Outatime, "Using speculation to enable low-latency continuous interaction for mobile cloud gaming," In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, pp. 151-165, 2015. DOI: http://doi.org/10.1145/2867070.2867076.
  5. Shahram Izadi et al., "Holoportation: Virtual 3D Teleportation in Real-time," Microsoft Research, 2016. DOI: http://doi.org/10.1145/2984511.2984517.
  6. Li Lin, Xiaofei Liao, Guang Tan, Hai Jin, Xiaobin Yang, Wei Zhang, and Bo Li, "LiveRender: A Cloud Gaming System Based on Compressed Graphics Streaming," Conference Paper in IEEE/ACM Transactions on Networking, 2014.
  7. Teemu Kamarainen, Matti Siekkinen, Jukka Eerikainen, Antti Yla-Jaaski, "CloudVR: Cloud Accelerated Interactive Mobile Virtual Reality," ACM Multimedia Conference, 2018. DOI: http://doi.org/10.1145/3240508.3240620.
  8. Andres Fuster-Guillo, Jorge Azorin-Lopez, Juan Miguel Castillo Zaragoza, Luis Fernando Perez Perez, Marcelo Saval-Calvo, and Robert B. Fisher, "3D Technologies to Acquire and Visualize the Human Body for Improving Dietetic Treatment," Proceedings, Vol. 31, No. 1, 2019. DOI: http://doi.org/10.3390/proceedings2019031053.
  9. J. Kovacs, "An Overview of Genlock," MicroImage Video Systems, 2001.
  10. Chenyang Zhang, Teng Huang, and Qiang Zhao, "A New Model of RGB-D Camera Calibration Based on 3D Control Field," Sensors, Vol. 19, No. 23, 2019. DOI: http://doi.org/10.3390/s19235082.
  11. Eisert, P. and Fechteler, P., "Low delay streaming of computer graphics," In Proceedings of the 15th IEEE International Conference on Image Processing, pp. 2704-2707, 2008. DOI: http://doi.org/10.1109/ICIP.2008.4712352.
  12. Duguet, F. and Drettakis, G., "Flexible point-based rendering on mobile devices," IEEE Trans. Comput. Graph. Appl., Vol. 24, No. 4, pp. 57-63, 2004. DOI: http://doi.org/10.1109/MCG.2004.5.
  13. Baratto, R. A., Kim, L. N., and Nieh, J., "Thinc: A virtual display architecture for thin-client computing," In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP'05). ACM, pp. 277-290, 2005. DOI: http://doi.org/10.1145/1095809.1095837.
  14. Lamberti, F. and Sanna, A., "A streaming-based solution for remote visualization of 3D graphics on mobile devices," IEEE Trans. Vis. Comput. Graph., Vol. 13, No. 2, pp. 247-260, 2007. DOI: http://doi.org/10.1109/TVCG.2007.29.
  15. Noimark, Y. and Cohen-Or, D., "Streaming scenes to mpeg-4 video-enabled devices," IEEE Comput. Graph. Appl., Vol. 23, pp. 58-64, 2003. DOI: http://doi.org/10.1109/MCG.2003.1159614.
  16. Alan Johnston, John Yoakum, and Kundan Singh, Avaya Inc, "Taking on WebRTC in an Enterprise" IEEE Communications Magazine, Vol. 51, pp. 48-54, 2013. DOI: http://doi.org/10.1109/MCOM.2013.6495760.
  17. WebRTC, https://webrtc.org/.
  18. Geonhee Lee, Pyeong-ho Choi, Hwa-seop Han, Seunghyun Lee, and Soonchul Kwon, "A Study on the Performance Comparison of 3D File Formats on the Web," International Journal of Advanced Smart Convergence (IJASC), Vol. 8, No. 1, pp. 65-74, 2019. DOI: http://doi.org/10.7236/IJASC.2019.8.1.65.
  19. Geonhee Lee, Seunghyun Lee, and Soonchul Kwon, "A study on Compression of 3D Model Data and Optimization of Website," Journal of Engineering and Applied Sciences (JEAS), Vol. 14, No. 1, pp. 3934- 3937, 2019. DOI: 10.3923/jeasci.2019.3934.3937.
  20. Duckkyoun Nam, Daehyeon Lee, Seunghyun Lee, and Soonchul Kwon, "Performance Compression of 3D File Formats on a Mobile Web Brower," International Journal of Broadcasting and Communication (IJIBC), Vol. 11, No. 2, pp. 31-42, 2019. DOI: http://doi.org/10.7236/IJIBC.2019.11.2.31.
  21. Duckkyoun Nam, Daehyeon Lee, Seunghyun Lee, and Soonchul Kwon, "A Comparative Study 3D Data Performance in Mobile Web Brosers in 4G and 5G Environments,," International Journal of Broadcasting and Communication (IJIBC), Vol. 11,No. 3,pp. 8-19, 2019. DOI: http://doi.org/10.7236/IJIBC.2019.11.3.8.