IEIE Transactions on Smart Processing and Computing

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

A Survey of Human Action Recognition Approaches that use an RGB-D Sensor

  • Farooq, Adnan (Department of Electronics and Electrical Engineering, Dongguk University -Seoul) ;
  • Won, Chee Sun (Department of Electronics and Electrical Engineering, Dongguk University -Seoul)
  • Received : 2015.06.20
  • Accepted : 2015.08.24
  • Published : 2015.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Human action recognition from a video scene has remained a challenging problem in the area of computer vision and pattern recognition. The development of the low-cost RGB depth camera (RGB-D) allows new opportunities to solve the problem of human action recognition. In this paper, we present a comprehensive review of recent approaches to human action recognition based on depth maps, skeleton joints, and other hybrid approaches. In particular, we focus on the advantages and limitations of the existing approaches and on future directions.

Keywords

References

  1. A. Veeraraghavan et al., "Matching shape sequences in video with applications in human movement analysis," Pattern Analysis and Machine Intelligence, IEEE Transactions, pp.1896-1909, Jun. 2004.
  2. W. Lin et al., "Human activity recognition for video surveillance," in Circuits and Systems, IEEE International Symposium on, pp. 2737-2740, May. 2008.
  3. H. S. Mojidra et al., "A Literature Survey on Human Activity Recognition via Hidden Markov Model," IJCA Proc. on International Conference on Recent Trends in Information Technology and Computer Science 2012 ICRTITCS, pp. 1-5, Feb. 2013.
  4. R. Gupta et al., "Human activities recognition using depth images," in Proc. of the 21st ACM international conference on Multimedia, pp. 283-292, Oct. 2013.
  5. Z. Zhang et al., "Microsoft kinect sensor and its effect." MultiMedia, IEEE, Vol. 19, No. 2, pp. 4-10, Feb. 2012.
  6. A. A. Chaaraoui, "Vision-based Recognition of Human Behaviour for Intelligent Environments," Director: Florez Revuelta, Franciso, Jan. 2014.
  7. M. Valera et al., "Intelligent distributed surveillance systems: a review," Vision, Image and Signal Processing, IEE Proceedings, Vol. 152, No. 2, pp. 192-204. Apr. 2005. https://doi.org/10.1049/ip-vis:20041147
  8. J. W. Hsieh et al., "Video-based human movement analysis and its application to surveillance systems," Multimedia, IEEE Transactions on, Vol. 10, No. 3, pp. 372-384, Apr. 2008. https://doi.org/10.1109/TMM.2008.917403
  9. V. Bloom et al., "G3d: A gaming action dataset and real time action recognition evaluation framework," Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, pp. 7-12, Jul. 2012.
  10. A. Fossati et al., "Consumer depth cameras for computer vision: research topics and applications," Springer Science & Business Media
  11. M. Parajuli et al., "Senior health monitoring using Kinect," Communications and Electronics (ICCE), Fourth International Conference on, pp. 309-312, Aug. 2012.
  12. C. Rougier et al., "Fall detection from depth map video sequences," Toward Useful Services for Elderly and People with Disabilities, Vol. 6719, pp. 121-128. Hun. 2011.
  13. W. Li et al., "Action recognition based on a bag of 3d points," Computer Vision and Pattern Recognition Workshops (CVPRW) IEEE Computer Society Conference on, pp. 9-14, Jun. 2010.
  14. J. Wang et al., "Mining actionlet ensemble for action recognition with depth cameras," Computer Vision and Pattern Recognition (CVPR) IEEE Conference on, pp. 1290-1297. Jun. 2012.
  15. L. Xia et al., "View invariant human action recognition using histograms of 3d joints," Computer Vision and Pattern Recognition Workshops (CVPRW) IEEE Computer Society Conference on, pp. 20-27, Jun. 2012.
  16. C. Ellis et al., "Exploring the trade-off between accuracy and observational latency in action recognition," International Journal of Computer Vision, Vol. 101, No. 3. Pp. 420-436. Aug. 2012. https://doi.org/10.1007/s11263-012-0550-7
  17. A. Shimada et al., "Kitchen scene context based gesture recognition: A contest in ICPR2012," Advances in Depth Image Analysis and Applications, Vol. 7854, pp. 168-185, Nov. 2011.
  18. A. W. Vieira et al., "Stop: Space-time occupancy patterns for 3d action recognition from depth map sequences," Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Vol. 7441, pp. 252-259. Sep. 2012.
  19. J. Wang et al., "Robust 3d action recognition with random occupancy patterns." 12th European Conference on Computer Vision, pp. 872-885. Oct. 2012.
  20. X. Yang et al., "Recognizing actions using depth motion maps-based histograms of oriented gradients," in Proc. of the 20th ACM international conference on Multimedia, pp. 1057-1060, Nov. 2012.
  21. N. Dalal et al., "Histograms of oriented gradients for human detection," Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, Vol. 1, pp. 886-893, Jun. 2005.
  22. A. Jalal et al., "Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home," Consumer Electronics, IEEE Transactions on, Vol. 58, No. 3, pp. 863-871, Aug. 2012. https://doi.org/10.1109/TCE.2012.6311329
  23. Y. Wang, K. Huang, and T. Tan. "Human activity recognition based on r transform." In Computer Vision and Pattern Recognition, IEEE Conference on, pp. 1-8. Jun. 2007.
  24. M. Z. Uddin et al., "Independent shape componentbased human activity recognition via Hidden Markov Model," Applied Intelligence, Vol. 33, No. 2, pp. 193-206. Jan. 2010. https://doi.org/10.1007/s10489-008-0159-2
  25. J. Han et al., "Human activity recognition in thermal infrared imagery," Computer Vision and Pattern Recognition. IEEE Computer Society Conference on, pp. 17, Jun. 2005.
  26. H. Othman et al., "A separable low complexity 2D HMM with application to face recognition," Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 25. No. 10, pp. 1229 - 1238, Oct. 2003. https://doi.org/10.1109/TPAMI.2003.1233897
  27. Y. Linde et al., "An algorithm for vector quantizer design," Communications, IEEE Transactions on, Vol. 28, No. 1, pp. 84-95, Jan. 1980. https://doi.org/10.1109/TCOM.1980.1094577
  28. O. Oreifej, & Z. Liu., "Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences," Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, 2013.
  29. L. Xia et al., "Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera," Computer Vision and Pattern Recognition, IEEE Conference on, pp. 2834-2841. Jun. 2013.
  30. Y. Song et al., "Body Surface Context: A New Robust Feature for Action Recognition from Depth Videos," Circuits and Systems for Video Technology, IEEE Transactions on, Vol. 24, No. 6, pp. 952-964, Jan. 2014. https://doi.org/10.1109/TCSVT.2014.2302558
  31. L. Xia et al., "View invariant human action recognition using histograms of 3d joints," Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, pp. 20-27. Jun. 2012.
  32. X. Yang et al., "Effective 3d action recognition using eigenjoints," Journal of Visual Communication and Image Representation, Vol. 25, No. 1, pp. 2-11, Jan. 2014. https://doi.org/10.1016/j.jvcir.2013.03.001
  33. O. Boiman et al., "In defense of nearest-neighbor based image classification," Computer Vision and Pattern Recognition, IEEE Conference on, pp. 1-8, Jun. 2008.
  34. G. Evangelidis et al., "Skeletal quads: Human action recognition using joint quadruples," Pattern Recognition (ICPR), 22nd International Conference on, pp. 4513-4518. Aug. 2014.
  35. J. Lei et al., "Fine-grained kitchen activity recognition using rgb-d." in Proc. of the ACM Conference on Ubiquitous Computing, pp. 208-211. Sep. 2012.
  36. S. Althloothi et al., "Human activity recognition using multi-features and multiple kernel learning," Pattern Recognition, Vol. 47. No. 5, pp. 1800-1812. May. 2014. https://doi.org/10.1016/j.patcog.2013.11.032
  37. M. Gonen and E. Alpaydin, "Multiple kernel learning algorithms," The Journal of Machine Learning Research, Vol. 12, pp. 2211-2268. Jan. 2011.

Cited by

  1. Depth map generation using a single image sensor with phase masks vol.24, pp.12, 2016, https://doi.org/10.1364/OE.24.012868
  2. Optical range-finding system using a single-image sensor with liquid crystal display aperture vol.41, pp.22, 2016, https://doi.org/10.1364/OL.41.005154