Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Semi-Supervised Deep Domain Adaptation Based Face Recognition Using Only a Single Training Sample

단일 훈련 샘플만을 활용하는 준-지도학습 심층 도메인 적응 기반 얼굴인식 기술 개발

  • Kim, Kyeong Tae (Division of Computer Engineering, Hankuk University of Foreign Studies) ;
  • Choi, Jae Young (Division of Computer Engineering, Hankuk University of Foreign Studies)
  • Received : 2022.06.27
  • Accepted : 2022.09.14
  • Published : 2022.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a semi-supervised domain adaptation solution to deal with practical face recognition (FR) scenarios where a single face image for each target identity (to be recognized) is only available in the training phase. Main goal of the proposed method is to reduce the discrepancy between the target and the source domain face images, which ultimately improves FR performances. The proposed method is based on the Domain Adatation network (DAN) using an MMD loss function to reduce the discrepancy between domains. In order to train more effectively, we develop a novel loss function learning strategy in which MMD loss and cross-entropy loss functions are adopted by using different weights according to the progress of each epoch during the learning. The proposed weight adoptation focuses on the training of the source domain in the initial learning phase to learn facial feature information such as eyes, nose, and mouth. After the initial learning is completed, the resulting feature information is used to training a deep network using the target domain images. To evaluate the effectiveness of the proposed method, FR performances were evaluated with pretrained model trained only with CASIA-webface (source images) and fine-tuned model trained only with FERET's gallery (target images) under the same FR scenarios. The experimental results showed that the proposed semi-supervised domain adaptation can be improved by 24.78% compared to the pre-trained model and 28.42% compared to the fine-tuned model. In addition, the proposed method outperformed other state-of-the-arts domain adaptation approaches by 9.41%.

Keywords

Acknowledgement

This research was supported by Hankuk University of Foreign Studies Research Fund. This research was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. 2021R1A2C1092322). This research was supported by the "Development of Wave Overtopping quantitative observation technology" funded by the Korea Institute of Marine Science & Technology Promotion(KIMST) (No. 20220180).

References

  1. P.J. Phillips, "A Cross Benchmark Assessment of a Deep Convolutional Neural Network for Face Recognition," Proceedings of the Twelfth IEEE International Conference on Automatic Face & Gesture Recognition, pp. 705-710, 2017
  2. K. Simonyan, and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv P reprint, arXiv:1409.1556, 2014.
  3. G.B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, "Labeled Faces in the Wild: Adatabase for Studying Face Recognition in Unconstrained Environments," Technical Report, 07-49, University of Massachusetts, Amherst, 2007.
  4. L. Wolf, T. Hassner, and I. Maoz, "Face Recognition in Unconstrained Videos with Matched Background Similarity," Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 529-534, 2011.
  5. P.J. Phillips, J.R. Beveridge, B.A. Draper, G. Givens, A.J. O'Toole, and D. Bolme, "The Good, the Bad, and the Ugly Facechallenge Problem," Image and Vision Computing. Vol. 30, No. 3, pp. 177-185, 2012. https://doi.org/10.1016/j.imavis.2012.01.004
  6. D. Yi, Z. Lei, S. Liao, and S.Z. Li, "Learning Face Representation from Scratch," arXiv P reprint, arXiv:1411.7923, 2014.
  7. P.J. Phillips, H. Moon, S.A. Rizvi, and P.J. Rauss, "The FERET Evaluation Methodology for Face Recognition Algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 10, pp. 1090-1104, 2000. https://doi.org/10.1109/34.879790
  8. R. Li, Q. Jiao, W. Cao, H.S. Wong, and S. Wu, "Model Adaptation: Unsupervised Domain Adaptation without Source Data," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9641-9650, 2020.
  9. G. He, X. Liu, F. Fan, and J. You, "Classification-Aware Semi-Supervised Domain Adaptation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 964-965, 2020.
  10. K. Saito, D. Kim, S. Sclaroff, T. Darrell, and K. Saenko, "Semi-Supervised Domain Adaptation via Minimax Entropy," Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8050-8058, 2019.
  11. A. Krizhevsky and G.E. Hinton, Learning Multiple Layers of Features from Tiny Images, Master Thesis, Department of Computer Science, University of Toronto, 2009.
  12. K. Bousmalis, G. Trigeorgis, N. Silberman, D. Krishnan, and D. Erhan, "Domain Separation Networks," Advances in Neural Information Processing Systems, pp. 343-351, 2016.
  13. Y. Zou, Z. Yu, B.V.K. Kumar, and J. Wang, "Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training," Proceedings of the European Conference on Computer Vision (ECCV), pp. 289-305, 2018.
  14. H. Guo, R. Pasunuru, and M. Bansal, "MultiSource Domain Adaptation for Text Classification via Distancenet-Bandits," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, No. 5, pp. 7830-7838, 2020.
  15. G. Matasci, M. Volpi, M. Kanevski, L. Bruzzone, and D. Tuia, "Semisupervised Transfer Component Analysis for Domain Adaptation in Remote Sensing Image Classification," IEEE Transactions on Geoscience and Remote Sensing, Vol. 53, No. 7, pp. 3550-3564, 2015. https://doi.org/10.1109/TGRS.2014.2377785
  16. C. Deng, X. Liu, C. Li, and D. Tao, "Active Multi-Kernel Domain Adaptation for Hyperspectral Image Classification," Pattern Recognition, Vol. 77, pp. 306-315, 2018. https://doi.org/10.1016/j.patcog.2017.10.007
  17. J. Peng, W. Sun, L. Ma, and Q. Du, "DiscrimiNative Transfer Joint Matching for Domain Adaptation in Hyperspectral Image Classification," IEEE Geoscience and Remote Sensing Letters, Vol. 16, No. 6, pp. 972-976, 2019. https://doi.org/10.1109/LGRS.2018.2889789
  18. W. Wang, R. Arora, K. Livescu, and J. Bilmes, "On Deep Multi-View Representation Learning," International Conference on Machine Learning, pp. 1083-1092, 2015.
  19. M. Long, Y. Cao, J. Wang, and M.I. Jordan, "Learning Transferable Features with Deep Adaptation Networks," Proceedings of the International Conference on International Conference on Machine Learning, pp. 97-105, 2015.
  20. M. Long, H. Zhu, J. Wang, and M.I. Jordan, "Deep Transfer Learning with Joint Adaptation Networks," International Conference on Machine Learning, pp. 2208-2217, 2017.
  21. Y. Ganin, E Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, and V. Lempitsky, "Domain-Adversarial Training of Neural Networks," The J ournal of Machine Learning Research, Vol. 17, No. 1, pp. 2096-2030, 2016.
  22. S. Hong, W. Im, J. Ryu, and H.S. Yang, "SSPP-DAN: Deep Domain Adaptation Network for Face Recognition with Single Sample per Person," IEEE International Conference on Image Processing (ICIP), pp. 825-829, 2017.
  23. Z. Luo, J. Hu, W. Deng, and H. Shen, "Deep Unsupervised Domain Adaptation for Face Recognition," 13th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 453-457, 2018.
  24. G. Wen, H. Chen, D. Cai, and X. He, "Improving Face Recognition with Domain Adaptation," Neurocomputing, Vol. 287, pp. 45-51, 2018. https://doi.org/10.1016/j.neucom.2018.01.079
  25. M. Wang and D. Weihong, "Deep Face Recognition with Clustering Based Domain Adaptation," Neurocomputing, Vol. 393, pp. 1-14, 2020. https://doi.org/10.1016/j.neucom.2020.02.005
  26. K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
  27. Z. Zhang and M. Sabuncu, "Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels," Advances in Neural Information Processing Systems 31 (NeurIPS 2018), 2018.
  28. E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, "Deep Domain Confusion: Maximizing for Domain Invariance," arXiv Preprint, arXiv:1412.3474, 2014.
  29. A. Gretton, K.M. Borgwardt, M.J. Rasch, B. Scholkopf, and A. Smola, "A Kernel TwoSample Test," The J ournal of Machine Learning Research, Vol. 13, No. 1, pp. 723-773, 2012.
  30. D.P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," arXiv Preprint, arXiv: 1412.6980, 2014.
  31. T.C. Kang and H.B. Kang, "Machine Learning Based Estimation of the Concentration of Fine Particulate Matter Using Domain Adaptation Method," Journal of Korea Multimedia Society, Vol. 20, No. 8, pp. 1208-1215, 2017. https://doi.org/10.9717/KMMS.2017.20.8.1208