KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Pose and Expression Invariant Alignment based Multi-View 3D Face Recognition

  • Ratyal, Naeem (Vision and Pattern Recognition Systems Research Group, Capital University of Science and Technology (CUST)) ;
  • Taj, Imtiaz (Vision and Pattern Recognition Systems Research Group, Capital University of Science and Technology (CUST)) ;
  • Bajwa, Usama (Department of Computer Science, COMSATS Institute of Information Technology) ;
  • Sajid, Muhammad (Vision and Pattern Recognition Systems Research Group, Capital University of Science and Technology (CUST))
  • Received : 2018.01.28
  • Accepted : 2018.04.28
  • Published : 2018.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, a fully automatic pose and expression invariant 3D face alignment algorithm is proposed to handle frontal and profile face images which is based on a two pass course to fine alignment strategy. The first pass of the algorithm coarsely aligns the face images to an intrinsic coordinate system (ICS) through a single 3D rotation and the second pass aligns them at fine level using a minimum nose tip-scanner distance (MNSD) approach. For facial recognition, multi-view faces are synthesized to exploit real 3D information and test the efficacy of the proposed system. Due to optimal separating hyper plane (OSH), Support Vector Machine (SVM) is employed in multi-view face verification (FV) task. In addition, a multi stage unified classifier based face identification (FI) algorithm is employed which combines results from seven base classifiers, two parallel face recognition algorithms and an exponential rank combiner, all in a hierarchical manner. The performance figures of the proposed methodology are corroborated by extensive experiments performed on four benchmark datasets: GavabDB, Bosphorus, UMB-DB and FRGC v2.0. Results show mark improvement in alignment accuracy and recognition rates. Moreover, a computational complexity analysis has been carried out for the proposed algorithm which reveals its superiority in terms of computational efficiency as well.

Keywords

References

  1. K. Bowyer, K. Chang and P. Flynn, "A survey of approaches and challenges in 3D and multimodal 3D+2D face recognition," Computer Vision and Image Understanding, vol. 101, no. 1, pp. 1-15, 2006. https://doi.org/10.1016/j.cviu.2005.05.005
  2. X. Wang, Q. Ruan, Y. Jin and G. An, "Three-dimensional face recognition under expression variation," EURASIP Journal on Image and Video Processing, 2014:51, 2014. https://doi.org/10.1186/1687-5281-2014-51
  3. L. Zhang, Z. Ding, H. Li, Y. Shen and J. Lu, "3D face recognition based on multiple keypoint descriptors and sparse representation," PLOS ONE, vol. 9, no. 6: e100120, 2014. https://doi.org/10.1371/journal.pone.0100120
  4. S. Elaiwat, M. Bennamoun, F. Boussaid and A. El-Sallam, "3-D face recognition using curvelet local features," IEEE Signal Processing Letters, vol. 21, pp. 172-175, 2014. https://doi.org/10.1109/LSP.2013.2295119
  5. S. Soltanpour, B. Boufama and Q.J.Wu, "A survey of local feature methods for 3D face recognition," Pattern Recognition, 2017.
  6. K. I. Chang, K. W. Bowyer and P. J. Flynn, "An evaluation of multimodal 2D+3D face biometrics," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 619-624, April 2005. https://doi.org/10.1109/TPAMI.2005.70
  7. C. BenAbdelkader and P. A. Griffin, "Comparing and combining depth and texture cues for face recognition," Image and Vision Computing, vol. 23, no. 3, pp. 339-352, 2005. https://doi.org/10.1016/j.imavis.2004.09.004
  8. C. Hesher, A. Srivastava and G. Erlebacher, "A novel technique for face recognition using range imaging," in Proc. of International Symposium on Signal Processing and Its Applications, vol. 2, pp. 201-204, Jul. 2003.
  9. H. Drira, B. Ben Amor, A. Srivastava, M. Daoudi and R. Slama, "3D face recognition under expressions, occlusions and pose variations," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 9, pp. 2270-2283, 2013. https://doi.org/10.1109/TPAMI.2013.48
  10. N. Alyuz, B. Gokberk and L. Akarun, "3-D face recognition under occlusion using masked projection," IEEE Transactions on Information Forensics and Security, vol. 8, pp. 789-802, 2013. https://doi.org/10.1109/TIFS.2013.2256130
  11. D. Smeets, J. Keustermans, D. Vandermeulen and P. Suetens, "MeshSIFT: local surface features for 3D face recognition under expression variations and partial data," Computer Vision and Image Understanding, vol. 117, no. 2, pp. 158-169, 2013. https://doi.org/10.1016/j.cviu.2012.10.002
  12. P. J. Besl and N. D. McKay, "A method for registration of 3-D shapes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, 1992. https://doi.org/10.1109/34.121791
  13. C. C. Queirolo, L. Silva, O. R. Bellon and M. P. Segundo, "3D face recognition using simulated annealing and the surface interpenetration measure," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 2, pp. 206-219, 2010. https://doi.org/10.1109/TPAMI.2009.14
  14. N. Alyuz, B. Gokberk and L. Akarun, "Regional registration for expression resistant 3-D face recognition," IEEE Transactions on Information Forensics and Security, vol. 5, no. 3, pp. 425- 440, Sep. 2010. https://doi.org/10.1109/TIFS.2010.2054081
  15. T. Papatheodorou and D. Rueckert, "3D face recognition," I-Tech Education and Publishing, Vienna 2007.
  16. Y. Wang, J. Liu and X. Tang, "Robust 3D face recognition by local shape difference boosting," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 10, pp. 1858-1870, 2010. https://doi.org/10.1109/TPAMI.2009.200
  17. A. B. Moreno and A. Sanchez, "GavabDB: a 3D face database," in Proc. of Second COST Workshop on Biometrics on the Internet: Fundamentals, Advances and Applications, pp. 77-82, 2004.
  18. A. Savran, N. Alyuz, H. Dibeklioglu, O. Celiktutan, B. Gokberk, B. Sankur and L. Akarun, "Bosphorus database for 3D face analysis," Biometrics and Identity Management, pp. 47-56, 2008.
  19. A. Colombo, C. Cusano and R. Schettini, "UMB-DB: A database of partially occluded 3D faces," in Proc. of International Conference on Computer Vision, pp. 2113-2119, 2011.
  20. P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min and W. Worek, "Overview of the face recognition grand challenge," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 947-954, 2005.
  21. S. Berretti, A. Del Bimbo and P.Pala, "3D face recognition using isogeodesic stripes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp. 2162-2177, 2010. https://doi.org/10.1109/TPAMI.2010.43
  22. F. R. Al-Osaimi, M. Bennamoun and A. Mian, "Integration of local and global geometrical cues for 3D face recognition," Pattern Recognition, vol. 41, no. 3, pp. 1030-1040, 2007. https://doi.org/10.1016/j.patcog.2007.07.009
  23. G. Chang and Y. Wang, "Robust 3D face recognition based on resolution invariant features," Pattern Recognition Letters, vol. 32, no. 7, pp. 1009-1019, 2011. https://doi.org/10.1016/j.patrec.2011.02.004
  24. S. Berretti, A. Del Bimbo and P. Pala, "Sparse matching of salient facial curves for recognition of 3D faces with missing parts," IEEE Transactions on Information Forensics and Security, vol. 8, pp. 374-389, 2012.
  25. M. H. Mahoor and M. Abdel-Mottaleb, "Face recognition based on 3D ridge images obtained from range data," Pattern Recognition, vol. 42, no. 3, pp. 445-451, 2009. https://doi.org/10.1016/j.patcog.2008.08.012
  26. D. Huang, M. Ardabilian, Y. Wang and L. Chen, "3-D face recognition using eLBP-based facial description and local feature hybrid matching," IEEE Transactions on Information Forensics and Security, vol. 7, no. 5, pp. 1551-1565, Oct. 2012. https://doi.org/10.1109/TIFS.2012.2206807
  27. W.Hariri, H.Tabia, N.Farah, A. Benouareth and D.Declercq, "3D face recognition using covariance based descriptors," Pattern Recognition Letters, vol.78, pp. 1-7, 2016. https://doi.org/10.1016/j.patrec.2016.03.028
  28. Y.Tang, H. Li, X.Sun, J. M.Morvan and L. Chen, " Principal curvature measures estimation and application to 3D face recognition," Journal of Mathematical Imaging and Vision, pp. 1-23, 2017.
  29. A.F. Abate, M. Nappi, D.Riccio and G. Sabatino, "2D and 3D face recognition: A survey," Pattern Recognition Letters, vol. 28, no. 14, pp. 1885-1906, 2007. https://doi.org/10.1016/j.patrec.2006.12.018
  30. V. Blanz and T. Vetter, "Face recognition based on fitting a 3D morphable model," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, 2003. https://doi.org/10.1109/TPAMI.2003.1227983
  31. T. Russ, C. Boehnen and T. Peters, "3D face recognition using 3D alignment for PCA," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1391- 1398, 2006.
  32. X. Lu and A.K. Jain, "Automatic feature extraction for multiview 3D face recognition," in Proc. of International Conference on Automatic Face and Gesture Recognition, pp. 585-590, April, 2006.
  33. P. Kamencay, R. Hudec, M. Benco and M. Zachariasova, "2D-3D face recognition method based on a modified CCA-PCA algorithm," International Journal of Advanced Robotic Systems, vol. 11, no. 36, pp. 1-8, 2014. https://doi.org/10.5772/56810
  34. S. Zafeiriou, G. A. Atkinson, M. F. Hansen, W.A. P. Smith,V. Argyriou, M.Petrou, M. L. Smith and L. N. Smith, "Face recognition and verification using photometric stereo: The photoface database and a comprehensive evaluation," IEEE Transactions on Information Forensics and Security, vol. 8, no. 1, pp. 121-135, 2013. https://doi.org/10.1109/TIFS.2012.2224109
  35. L. Spreeuwers, "Fast and accurate 3D face recognition using registration to an intrinsic coordinate system and fusion of multiple region classifiers," International Journal of Computer Vision, vol. 93, pp. 389-414, 2011. https://doi.org/10.1007/s11263-011-0426-2
  36. S. Jahanbin, R. Jahanbin and A. C. Bovik, "Passive three dimensional face recognition using isogeodesic contours and procrustes analysis," International Journal of Computer Vision, vol. 105, pp. 87-108, 2013. https://doi.org/10.1007/s11263-013-0631-2
  37. X. Peng, M. Bennamoun and A. S. Mian, "A training-free nose tip detection method from face range images," Pattern Recognition, vol. 44, pp. 544-558, 2011. https://doi.org/10.1016/j.patcog.2010.09.015
  38. G. G. Slabaugh, "Computing Euler angles from a rotation matrix," Technical Report, 1999. Available at: (Date of access: 07-April-2016).
  39. U. I. Bajwa, I. A. Taj and M. W. Anwar, "A unified classifier for robust face recognition based on combining multiple subspace algorithms," Optics Communications, vol. 285, no. 21-22, pp. 4324-4332, Oct. 2012. https://doi.org/10.1016/j.optcom.2012.07.036
  40. A. A. Ross, A. K. Jain and K. Nandakumar, "Handbook of Multibiometrics (Levels of fusion in biometrics)," Springer, US, 2006.
  41. A. A. Ross, A. K. Jain and K. Nandakumar, "Handbook of Multibiometrics (Score level fusion)," Springer, US, 2006.
  42. U. I. Bajwa, I. A. Taj, M. W. Anwar and X. Wang, "A multifaceted independent performance analysis of facial subspace recognition algorithms," PLOS ONE, vol. 8, no. 2:e56510, 2013. https://doi.org/10.1371/journal.pone.0056510
  43. O. Sagi, L. Rokach, "Ensemble learning: A survey," Wires Data Mining and Knowledge Discovery, e1249, 2018.
  44. A. S. Mian, M. Bennamoun and R. Owens, "An efficient multimodal 2D-3D hybrid approach to automatic face recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1927-1943, 2007. https://doi.org/10.1109/TPAMI.2007.1105
  45. X. Li, T. Jia and H. Zhang, "Expression-insensitive 3D face recognition using sparse representation," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2575-2582, 2009.
  46. S. Berretti, N. Werghi, A. Del Bimbo and P. Pala, "Matching 3D face scans using interest points and local histogram descriptors," Computers and Graphics, vol. 37, no. 5, pp. 509-525, 2013. https://doi.org/10.1016/j.cag.2013.04.001
  47. H. Li, D. Huang, J.M. Morvan,Y. Wang and L. Chen , "Towards 3D face recognition in the real: a registration-free approach using fine-grained matching of 3D keypoint descriptors," International Journal of Computer Vision, vol. 113, no. 2, pp. 128-142, 2015. https://doi.org/10.1007/s11263-014-0785-6
  48. S.Z. Gilani, A. Mian and P. Eastwood, "Deep, dense and accurate 3D face correspondence for generating population specific deformable models," Pattern Recognition , vol. 69, pp. 238-250 , 2017. https://doi.org/10.1016/j.patcog.2017.04.013

Cited by

  1. Content-Based Image Retrieval and Feature Extraction: A Comprehensive Review vol.2019, pp.None, 2018, https://doi.org/10.1155/2019/9658350
  2. Human Posture Recognition and Estimation Method Based on 3D Multiview Basketball Sports Dataset vol.2021, pp.None, 2018, https://doi.org/10.1155/2021/6697697