International Journal of Internet, Broadcasting and Communication

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Real-Time Arbitrary Face Swapping System For Video Influencers Utilizing Arbitrary Generated Face Image Selection

  • Jihyeon Lee (Dept. Artificial Intelligence Appliance, Dongseo University) ;
  • Seunghoo Lee (Dept. Artificial Intelligence Appliance, Dongseo University) ;
  • Hongju Nam (Dept. Artificial Intelligence Appliance, Dongseo University) ;
  • Suk-Ho Lee (Dept. Computer Engineering, Dongseo University)
  • Received : 2023.02.12
  • Accepted : 2023.02.17
  • Published : 2023.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper introduces a real-time face swapping system that enables video influencers to swap their faces with arbitrary generated face images of their choice. The system is implemented as a Django-based server that uses a REST request to communicate with the generative model,specifically the pretrained stable diffusion model. Once generated, the generated image is displayed on the front page so that the influencer can decide whether to use the generated face or not, by clicking on the accept button on the front page. If they choose to use it, both their face and the generated face are sent to the landmark extraction module to extract the landmarks, which are then used to swap the faces. To minimize the fluctuation of landmarks over time that can cause instability or jitter in the output, a temporal filtering step is added. Furthermore, to increase the processing speed the system works on a reduced set of the extracted landmarks.

Keywords

Acknowledgement

This work was supported by Dongseo University, "Dongseo Cluster Project" Research Fund of 2022(DSU-20220001).

References

  1. I. Dunham, (2020). "Faceless Youtubers: How Content Creators Shape Audience Expectation," in Proc. The 21st Annual Conference of the Association of Internet Researchers, Oct. 27-31, 2020. DOI:https://doi.org/10.5210/spir.v2020i0.11205
  2. C. Clark-Gordon, N. Bowman, A. Goodboy, A. Wright, (2019) "Anonymity and Online Self-Disclosure: A Meta-Analysis," Communication Reports, Vol. 32, Issue 2, pp.98-111, DOI: https://doi.org/10.1080/08934215.2019.1607516
  3. G. David, J. Edward , "Deepfake Video Detection Using Recurrent Neural Networks," Proc. of 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Nov. 27-30, 2018. DOI: https://doi.org/10.1109/AVSS.2018.8639163
  4. L. Yuezun, L. Siwei, "Exposing DeepFake Videos By Detecting Face Warping Artifacts," Proc. of Conference on Computer Vision and Pattern Recognition(CVPR 2019), pp. 46-52, June 16-20, 2019. DOI: https://doi.org/10.48550/arXiv.1811.00656
  5. Reface. https://reface.app.
  6. Snap Camera. https://snapcamera.snapchat.com
  7. Face Swap Live. https://faceswaplive.com
  8. MSQRD. https://msqrd.kr.uptodown.com
  9. DeepArt.io. https://creativitywith.ai/deepartio/
  10. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, "High-Resolution Image Synthesis with Latent Diffusion Models," Proc. of the International Conference on Computer Vision and Pattern Recognition (CVPR 2022), pp. 10684-10695, June 18-24, DOI: https://doi.org/10.1109/CVPR52688.2022.01042
  11. Mediapipe. https://google.github.io/mediapipe
  12. Django rest framework. https://www.django-rest-framework.org
  13. W. Enders, Stationary Time-Series Models, Applied Econometric Time Series (Second ed.). Wiley. pp. 48-107, 2004
  14. Pysource. https://pysource.com