KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Face Identification Using a Near-Infrared Camera in a Nonrestrictive In-Vehicle Environment

적외선 카메라를 이용한 비제약적 환경에서의 얼굴 인증

  • 기민송 (연세대학교 컴퓨터과학과) ;
  • 최영우 (숙명여자대학교 소프트웨어학부)
  • Received : 2020.10.22
  • Accepted : 2020.11.29
  • Published : 2021.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

There are unrestricted conditions on the driver's face inside the vehicle, such as changes in lighting, partial occlusion and various changes in the driver's condition. In this paper, we propose a face identification system in an unrestricted vehicle environment. The proposed method uses a near-infrared (NIR) camera to minimize the changes in facial images that occur according to the illumination changes inside and outside the vehicle. In order to process a face exposed to extreme light, the normal face image is changed to a simulated overexposed image using mean and variance for training. Thus, facial classifiers are simultaneously generated under both normal and extreme illumination conditions. Our method identifies a face by detecting facial landmarks and aggregating the confidence score of each landmark for the final decision. In particular, the performance improvement is the highest in the class where the driver wears glasses or sunglasses, owing to the robustness to partial occlusions by recognizing each landmark. We can recognize the driver by using the scores of remaining visible landmarks. We also propose a novel robust rejection and a new evaluation method, which considers the relations between registered and unregistered drivers. The experimental results on our dataset, PolyU and ORL datasets demonstrate the effectiveness of the proposed method.

차량 내부에는 조명 변화, 부분적인 가림 및 운전자의 상태 변화와 같은 제한되지 않은 조건들이 존재한다. 본 논문에서는 비 제약적인 차량 환경에서의 운전자 얼굴 인증 시스템을 제안한다. 제안한 방법은 차량 내부 및 외부의 조명 변화에 따라 발생하는 얼굴 이미지의 변화를 최소화하기 위해서 근적외선(NIR) 카메라를 사용한다. 특히 정면에서의 강한 빛에 노출된 얼굴 이미지를 처리하기 위해서, 학습 이미지의 평균과 분산을 사용하여 정상적인 얼굴 이미지로부터 빛에 과다하게 노출된 이미지로 변환하여 사용한다. 따라서 정상적인 조명에서의 얼굴 분류기와 강한 정면광에서의 얼굴 분류기를 각각 동시에 만들어진다. 제안하는 얼굴 분류기는 얼굴 랜드마크를 추출하고 각 랜드마크의 신뢰도 점수를 합산하여 얼굴을 최종적으로 식별한다. 특히 각 랜드마크를 인식하여 부분적인 얼굴 가림에 강하기 때문에 안경이나 선글라스를 착용하는 상황에서도 높은 성능 향상이 가능하다. 즉 가려지지 않은 남은 랜드마크의 점수를 사용하여 운전자를 인식할 수 있다. 또한 등록 운전자와 미등록 운전자 간의 관계를 고려한 새로운 인식 거부 방법과 새로운 평가 방법을 논문에서 제안한다. 자체 취득한 데이터 셋, 공인된 PolyU 및 ORL 데이터 셋으로 실험한 결과 제안한 방법이 효과적임을 확인할 수 있었다.

Keywords

References

  1. Q. Bi, M. Yang, C. Wang, and B. Wang, "An efficient hierarchical convolutional neural network for traffic object detection," in Proceedings of IEEE Intelligent Vehicles Symposium (IV), 2018.
  2. K. Lim, Y. Hong, M. Ki, Y. Choi, and H. Byun, "Vision-based recognition of road regulation for intelligent vehicle," in 2018 IEEE Intelligent Vehicles Symposium (IV), pp.1418-1425, 2018.
  3. Z. Yang, J. Li, and H. Li, "Real-time pedestrian and vehicle detection for autonomous driving," in Proceedings of IEEE Intelligent Vehicles Symposium (IV), 2018.
  4. N. Crosswhite, J. Byrne, C. Stauffer, O. Parkhi, Q. Cao, and A. Zisserman, "Template adaptation for face verification and identification," Image and Vision Computing, 2018.
  5. E. M. Cherrat, R. Alaoui, and H. Bouzahir, "Convolutional neural networks approach for multimodal biometric identification system using the fusion of fingerprint, finger-vein and face images," PeerJ Computer Science, 2020.
  6. S. Farokhi, S. M. Shamsuddin, U. Sheikh, J. Flusser, M. Khansari, and K. Jafari-Khouzani, "Near infrared face recognition by combining zernike moments and undecimated discrete wavelet transform," Digital Signal Processing, 2014.
  7. B. Zhang, L. Zhang, D. Zhang, and L. Shen, "Directional binary code with application to polyu near-infrared face database," Pattern Recognition Letters, 2010.
  8. K. Pan, S. Liao, Z. Zhang, S. Z. Li, and P. Zhang, "Part-based face recognition using near infrared images," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007.
  9. T. Ojala, M. Pietikainen, and D. Harwood, "A comparative study of texture measures with classification based on featured distributions," Pattern Recognition, 1996.
  10. ATT Laboratories Cambridge, The orl database of faces, 1994.
  11. M. Ki, B. Cho, T. Jeon, Y. Choi, and H. Byun, "Face Identification for an in-vehicle Surveillance System Using Near Infrared Camera," in 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp.1-6, 2018.
  12. D. Chen, X. Cao, F. Wen, and J. Sun, "Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.3025-3032, 2013.
  13. Y. Taigman, M. Yang, M. A. Ranzato, and L. Wolf, "Deepface: Closing the gap to human-level performance in face verification," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1701-1708, 2014.
  14. F. Schroff, D. Kalenichenko, and J. Philbin, "Facenet: A unified embedding for face recognition and clustering," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.815-823, 2014.
  15. S. Ren, X. Cao, Y. Wei, and J. Sun, "Face alignment at 3000 fps via regressing local binary features," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1685-1692, 2014.
  16. M. Turk and A. Pentland, "Eigenfaces for recognition," Journal of Cognitive Neuroscience, Vol.3, No.1, pp.71-86, 1991. https://doi.org/10.1162/jocn.1991.3.1.71
  17. S. Z. Li, R. Chu, S. Liao, and L. Zhang, "Illumination invariant face recognition using near-infrared images," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.29, No.4, pp.627-639, 2007. https://doi.org/10.1109/TPAMI.2007.1014
  18. S. Farokhi, U. Sheikh, J. Flusser, and B. Yang, "Near infrared face recognition using Zernike moments and Hermite kernels," Information Sciences, Vol.316, pp.234-245, 2015. https://doi.org/10.1016/j.ins.2015.04.030
  19. X. Zhang, M. Peng, and T. Chen, "Face recognition from near-infrared images with convolutional neural network," in 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP), 2016.
  20. X. Luo, Y. Xu and J. Yang, "Multi-resolution dictionary learning for face recognition," Pattern Recognition, Vol.93, pp.283-292, 2019. https://doi.org/10.1016/j.patcog.2019.04.027
  21. D. Wang and S. Kong "A classification-oriented dictionary learning model: Explicitly learning the particularity and commonality across categories," Pattern Recognition, 2014.
  22. J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, "Robust face recognition via sparse representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
  23. Z. Jiang, Z. Lin, and L. Davis, "Label consistent K-SVD: Learning a discriminative dictionary for recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013.