ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

MixFace: Improving face verification with a focus on fine-grained conditions

  • Junuk Jung (School of Computer Science and Engineering, Korea University of Technology and Education) ;
  • Sungbin Son (School of Computer Science and Engineering, Korea University of Technology and Education) ;
  • Joochan Park (School of Computer Science and Engineering, Korea University of Technology and Education) ;
  • Yongjun Park (School of Computer Science and Engineering, Korea University of Technology and Education) ;
  • Seonhoon Lee (School of Computer Science and Engineering, Korea University of Technology and Education) ;
  • Heung-Seon Oh (School of Computer Science and Engineering, Korea University of Technology and Education)
  • Received : 2023.06.01
  • Accepted : 2023.11.03
  • Published : 2024.08.20
Download PDF
⟨ Previous Next ⟩

Abstract

The performance of face recognition (FR) has reached a plateau for public benchmark datasets, such as labeled faces in the wild (LFW), celebrities in frontal-profile in the wild (CFP-FP), and the first manually collected, in-the-wild age database (AgeDB), owing to the rapid advances in convolutional neural networks (CNNs). However, the effects of faces under various fine-grained conditions on FR models have not been investigated, owing to the absence of relevant datasets. This paper analyzes their effects under different conditions and loss functions using K-FACE, a recently introduced FR dataset with fine-grained conditions. We propose a novel loss function called MixFace, which combines classification and metric losses. The superiority of MixFace in terms of effectiveness and robustness was experimentally demonstrated using various benchmark datasets.

Keywords

Acknowledgement

This paper was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIT) (No. NRF-2019R1G1A1003312) and (No. NRF-2021R1I1A3052815).

References

  1. K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, (IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA), June 27-30, 2016, 2016, pp. 770-778.
  2. J. Hu, L. Shen, and G. Sun, Squeeze-and-excitation networks, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA), 2018, pp. 7132-7141.
  3. K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint, 2014, DOI 10.48550/arXiv.1409.1556.
  4. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, Going deeper with convolutions, (IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA), June 7-12, 2015, 2015, pp. 1-9.
  5. F. Wang, L. Chen, C. Li, S. Huang, Y. Chen, C. Qian, and C. C. Loy, The devil of face recognition is in the noise, (Proceedings of the European Conference on Computer vision (ECCV), Munich, Germany), 2018, pp. 765-780.
  6. J. Deng, J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou, Retina-Face: Single-shot multi-level face localisation in the wild, (Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA), 2020, pp. 5203-5212.
  7. Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, VGGFace2: A dataset for recognising faces across pose and age, (13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xian, China), 2018, pp. 67-74.
  8. S. Chopra, R. Hadsell, and Y. LeCun, Learning a similarity metric discriminatively, with application to face verification, (IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, USA), 2005, pp. 539-546.
  9. R. Hadsell, S. Chopra, and Y. LeCun, Dimensionality reduction by learning an invariant mapping, (IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), New York, USA), 2006, pp. 1735-1742.
  10. O. M. Parkhi, A. Vedaldi, and A. Zisserman, Deep face recognition, (Proceedings of the British Machine Vision Conference 2015, Swansea, UK), Sept., 2015, 2015, pp. 41.1-41.12.
  11. F. Schroff, D. Kalenichenko, and J. Philbin, FaceNet: A unified embedding for face recognition and clustering, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815-823.
  12. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, Deepface: Closing the gap to human-level performance in face verification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA), 2014, pp. 1701-1708.
  13. Y. Wen, K. Zhang, Z. Li, and Y. Qiao, A discriminative feature learning approach for deep face recognition, (European Conference on Computer Vision, Amsterdam, Netherlands), 2016, pp. 499-515.
  14. K. Sohn, Improved deep metric learning with multi-class n-pair loss objective, (Proceedings of the 30th International Conference on Neural Information Processing Systems), 2016, pp. 1857-1865.
  15. X. Wang, X. Han, W. Huang, D. Dong, and M. R. Scott, Multi-similarity loss with general pair weighting for deep metric learning, (Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA) 2019, pp. 5022-5030.
  16. J. Deng, J. Guo, N. Xue, and S. Zafeiriou, ArcFace: Additive angular margin loss for deep face recognition, (Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA), 2019, pp. 4690-4699.
  17. W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song, SphereFace: Deep hypersphere embedding for face recognition, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA), 2017, pp. 212-220.
  18. F. Wang, J. Cheng, W. Liu, and H. Liu, Additive margin softmax for face verification, IEEE Sig. Process. Lett. 25 (2018), no. 7, 926-930. https://doi.org/10.1109/LSP.2018.2822810
  19. H. Wang, Y. Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, and W. Liu, CosFace: Large margin cosine loss for deep face recognition, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA), 2018, pp. 5265-5274.
  20. W. Ge, Deep metric learning with hierarchical triplet loss, (Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany), 2018, pp. 269-285.
  21. B. Harwood, V. Kumar BG, G. Carneiro, I. Reid, and T. Drummond, Smart mining for deep metric learning, (Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy), 2017, pp. 2821-2829.
  22. B. Yu and D. Tao, Deep metric learning with tuplet margin loss, (Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Rep. of Korea), 2019, pp. 6490-6499.
  23. X. Zhang, R. Zhao, Y. Qiao, X. Wang, and H. Li, AdaCos: Adaptively scaling cosine logits for effectively learning deep face representations, (Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA), 2019, pp. 10823-10832.
  24. G. B. Huang, M. Mattar, T. Berg, and E. Learned-Miller, Labeled faces in the wild: A database forstudying face recognition in unconstrained environments, (Workshop on Faces in 'Real-Life' Images: Detection, Alignment, and Recognition, Seoul, Rep. of Korea), 2008.
  25. S. Sengupta, J.-C. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs, Frontal to profile face verification in the wild, (IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA), 2016, pp. 1-9.
  26. S. Moschoglou, A. Papaioannou, C. Sagonas, J. Deng, I. Kotsia, and S. Zafeiriou, AgeDB: The first manually collected, in-the-wild age database, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA), 2017, pp. 51-59.
  27. Y. Choi, H. Park, G. P. Nam, H. Kim, H. Choi, J. Cho, and I.-J. Kim, K-FACE: A large-scale kist face database in consideration with unconstrained environments, arXiv preprint, 2021, DOI 10.48550/arXiv.2103.02211.
  28. Y. Sun, C. Cheng, Y. Zhang, C. Zhang, L. Zheng, Z. Wang, and Y. Wei, Circle loss: A unified perspective of pair similarity optimization, (Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA), 2020, pp. 6398-6407.
  29. J. Bergstra and Y. Bengio, Random search for hyper-parameter optimization, J. Mach. Learn. Res. 13 (2012), no. 2, 281-305.
  30. Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, MS-Celeb-1M: S dataset and benchmark for large-scale face recognition, (European Conference on Computer Vision, Amsterdam, Netherlands), 2016, pp. 87-102.
  31. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, MobileNetV2: Inverted residuals and linear bottlenecks, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT. USA), 2018, pp. 4510-4520.