Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Generation of Masked Face Image Using Deep Convolutional Autoencoder

컨볼루션 오토인코더를 이용한 마스크 착용 얼굴 이미지 생성

  • Lee, Seung Ho (Department of Future Technology, Korea University of Technology and Education)
  • Received : 2022.04.10
  • Accepted : 2022.04.29
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Researches of face recognition on masked faces have been increasingly important due to the COVID-19 pandemic. To realize a stable and practical recognition performance, large amount of facial image data should be acquired for the purpose of training. However, it is difficult for the researchers to obtain masked face images for each human subject. This paper proposes a novel method to synthesize a face image and a virtual mask pattern. In this method, a pair of masked face image and unmasked face image, that are from a single human subject, is fed into a convolutional autoencoder as training data. This allows learning the geometric relationship between face and mask. In the inference step, for a unseen face image, the learned convolutional autoencoder generates a synthetic face image with a mask pattern. The proposed method is able to rapidly generate realistic masked face images. Also, it could be practical when compared to methods which rely on facial feature point detection.

코로나19 팬데믹으로 인해 마스크 착용이 일상화되면서 마스크 착용 얼굴을 식별하는 얼굴인식 연구에 대한 중요도가 높아지고 있다. 안정된 얼굴인식 성능을 위해서는 인식 대상에 대한 풍부한 학습용 이미지 확보가 필요하지만 인물 별로 마스크 착용 얼굴 이미지를 다량 확보하는 것은 쉽지 않다. 본 논문에서는 마스크 미착용 얼굴 이미지에 가상의 마스크 패턴을 합성하는 새로운 방법을 제안한다. 제안 방법은 동일 인물에 대해 마스크 미착용 얼굴 이미지와 마스크 착용 얼굴 이미지를 쌍으로 컨볼루션 오토인코더에 입력하여 얼굴과 마스크의 기하학적 관계를 학습한다. 학습이 완료된 컨볼루션 오토인코더는 학습에 사용되지 않은 새로운 마스크 미착용 얼굴 이미지에 가상의 마스크 패턴을 자연스러운 형태로 합성해준다. 제안 방법은 고속으로 대량의 마스크 착용 얼굴 이미지를 생성할 수 있으며, 얼굴 특징점 추출에 기반하는 마스크 합성 방법에 비해 실용적이다.

Keywords

References

  1. M. L. Ngan, P. J. Grother, and K. K. Hanaoka, "Ongoing Face Recognition Vendor Test (FRVT) Part 6A: Face Recognition Accuracy with Masks Using Pre-COVID-19 Algorithms," National Institute of Standards and Technology, Technical Report NISTIR 3811, Jul. 2020.
  2. H. I. Kim, J. Y. Moon, and J. Y. Park, "Research Trends for Deep Learning-Based High-Performance Face Recognition Technology," Electronics and Telecommunications Trends, vol. 33, no. 4, pp. 43-53, Aug. 2018. https://doi.org/10.22648/ETRI.2018.J.330405
  3. Y. Kim, X. Zhang, and J. Park, "Automatic Mask Face Data Synthesis System," in Proceedings of Conference of the Korean Institute of Broadcast and Media Engineers, Online, pp. 239-240, 2020.
  4. OpenCV-Python [cited 2022 April 10], [Internet]. Available: https://docs.opencv.org/4.3.0/.
  5. J. Masci, U. Meier, D. Cires,an, and J. Schmidhuber, "Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction," in Proceedings of International Conference on Artificial Neural Networks, Espoo, Finland, pp. 52-59, 2011.
  6. H. Kim, S. Lee, and Y. No, "Face Recognition in the Wild," Information and Communications Magazine, vol. 31, no. 4, pp. 88-98, Mar. 2014.
  7. G. Hua, M. H. Yang, E. Learned-Miller, Y. Ma, M. Turk, D. J. Kriegman, and T. S. Huang, "Introduction to the Special Section on Real-world Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1921-1924, Oct. 2011. https://doi.org/10.1109/TPAMI.2011.182
  8. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient Based Learning Applied to Document Recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998. https://doi.org/10.1109/5.726791
  9. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," in Proceedings of Advances in Neural Information Processing Systems, Lake Taho: NV, USA, pp. 1097-1105, 2012.
  10. SNOW Face Recognition Application, [Internet]. Available: https://play.google.com/store/apps/details?id=com.campmobile.snow&hl=ko&gl=US.
  11. GTAV Face Database [cited 2022 April 10], [Internet]. Available: https://gtav.upc.edu/en/research-areas/face-database.
  12. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," in Proceedings of Advances Neural Information Processing Systems Conference, Montreal, Canada, pp. 2672-2680, 2014.