International Journal of Internet, Broadcasting and Communication

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Real-time Markerless Facial Motion Capture of Personalized 3D Real Human Research

  • Received : 2021.12.08
  • Accepted : 2021.12.16
  • Published : 2022.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Real human digital models appear more and more frequently in VR/AR application scenarios, in which real-time markerless face capture animation of personalized virtual human faces is an important research topic. The traditional way to achieve personalized real human facial animation requires multiple mature animation staff, and in practice, the complex process and difficult technology may bring obstacles to inexperienced users. This paper proposes a new process to solve this kind of work, which has the advantages of low cost and less time than the traditional production method. For the personalized real human face model obtained by 3D reconstruction technology, first, use R3ds Wrap to topology the model, then use Avatary to make 52 Blend-Shape model files suitable for AR-Kit, and finally realize real-time markerless face capture 3D real human on the UE4 platform facial motion capture, this study makes rational use of the advantages of software and proposes a more efficient workflow for real-time markerless facial motion capture of personalized 3D real human models, The process ideas proposed in this paper can be helpful for other scholars who study this kind of work.

Keywords

References

  1. Legde K. "Projecting motion capture: designing and implementing a modular and flexible facial animation pipeline to evaluate different perceptual effects". BTU Cottbus-Senftenberg, 2021. DOI: 10.1145/2811265
  2. Eloise Berson, Catherine Soladie, and Nicolas Stoiber. "Real-Time Cleaning and Refinement of Facial Animation Signals". In Proceedings of the 2020 The 4th International Conference on Graphics and Signal Processing. Association for Computing Machinery, New York, NY, USA, pp. 70-75. 2020. Doi: https://doi.org/10.1145/3406971.3406985
  3. Intyanto, G. W., D. A. F. Yuniarti, and A. S. Pawening. "Implement of the transformation for markerless facial motion capture in 3D animation." Journal of Physics: Conference Series. Vol. 1836. No. 1. IOP Publishing, 2021. Doi: 10.1088/1742-6596/1836/1/012033
  4. M. E. Latoschik, F. Kern, J. -P. Stauffert, A. Bartl, M. Botsch and J. -L. Lugrin, "Not Alone Here?! Scalability and User Experience of Embodied Ambient Crowds in Distributed Social Virtual Reality," in IEEE Transactions on Visualization and Computer Graphics, vol. 25, no. 5, pp. 2134-2144, May 2019, Doi: 10.1109/TVCG.2019.2899250.
  5. Christian Murphy, Sudhir Mudur, Daniel Holden, Marc-Andre Carbonneau, Donya Ghafourzadeh, Andre Beauchamp, "Artist guided generation of video game production quality face textures." Computers & Graphics, Volume 98, pp. 268-279. 2021. https://doi.org/10.1016/j.cag.2021.06.004.
  6. Aidan P. Murphy, David A. Leopold, "A parameterized digital 3D model of the Rhesus macaque face for investigating the visual processing of social cues." Journal of Neuroscience Methods, Volume 324, 2019, https://doi.org/10.1016/j.jneumeth.2019.06.001.
  7. Patterson E, Baron J, Simpson D. "Landmark-Based Re-topology of Stereo-Pair Acquired Face Meshes". International Conference on Computer Vision and Graphics. Springer, Cham, pp. 425-437. 2018. Doi: 10.1007/978-3-030-00692-1_37
  8. Gu Jie, Juwon Hwang, Choi, Chulyoung. "Case Study of Short Animation with Facial Capture Technology Using Mobile." International Journal of Internet, Broadcasting and Communication. Vol.12 No.3 pp. 56-63. 2020. http://dx.doi.org/10.7236/IJIBC.2020.12.3.56
  9. Sutanto Edward Budiman and Suk-Ho Lee, "Virtual Reality Image Shooting for Single Person Broadcasting with Multiple Smartphones." International Journal of Internet, Broadcasting and Communication. Vol.11 No.2. pp. 43-49. 2019. http://dx.doi.org/10.7236/IJIBC.2019.11.2.43
  10. Daehyeon Lee, Munyong Lee Sang-ha Lee, Jaehyun Lee, Soonchul Kwon, "Implementation of AR Remote Rendering Techniques for Real-time Volumetric 3D Video." International Journal of Internet, Broadcasting and Communication. Vol.12 No.2. pp. 90-97. 2020. http://dx.doi.org/10.7236/IJIBC.2020.12.2.90
  11. Kyu-Ha Kim, Sang-Hyun Lee, "Development of Structured Light 3D Scanner Based on Image Processing." International Journal of Internet, Broadcasting and Communication. Vol.11 No.4. pp. 49-58. 2019. http://dx.doi.org/10.7236/IJIBC.2019.11.4.49
  12. Bongkyu Lee, "A Study on the Analysis of Structural Textures using CNN (Convolution Neural Network)." The Journal of The Institute of Internet, Broadcasting and Communication (IIBC). Vol. 20, No. 4, pp. 201-205, 2020 https://doi.org/10.7236/JIIBC.2020.20.4.201