Browse > Article
http://dx.doi.org/10.3837/tiis.2022.12.012

Enhanced 3D Residual Network for Human Fall Detection in Video Surveillance  

Li, Suyuan (School of Computer Science and Engineering, Northeastern University)
Song, Xin (School of Computer Science and Engineering, Northeastern University)
Cao, Jing (School of Computer Science and Engineering, Northeastern University)
Xu, Siyang (School of Computer Science and Engineering, Northeastern University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.12, 2022 , pp. 3991-4007 More about this Journal
Abstract
In the public healthcare, a computational system that can automatically and efficiently detect and classify falls from a video sequence has significant potential. With the advancement of deep learning, which can extract temporal and spatial information, has become more widespread. However, traditional 3D CNNs that usually adopt shallow networks cannot obtain higher recognition accuracy than deeper networks. Additionally, some experiences of neural network show that the problem of gradient explosions occurs with increasing the network layers. As a result, an enhanced three-dimensional ResNet-based method for fall detection (3D-ERes-FD) is proposed to directly extract spatio-temporal features to address these issues. In our method, a 50-layer 3D residual network is used to deepen the network for improving fall recognition accuracy. Furthermore, enhanced residual units with four convolutional layers are developed to efficiently reduce the number of parameters and increase the depth of the network. According to the experimental results, the proposed method outperformed several state-of-the-art methods.
Keywords
Video surveillance; fall detection; deep learning; residual network; 3D CNN;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 M. Li, G. Xu, B. He, X. Ma and J. Xie, "Pre-Impact Fall Detection Based on a Modified Zero Moment Point Criterion Using Data from Kinect Sensors," IEEE Sensors Journal, vol. 18, no. 13, pp. 5522-5531, July. 2018.   DOI
2 B. Kwolek and M. Kepski, "Human fall detection on embedded platform using depth maps and wireless accelerometer," Comput. Methods Programs Biomed, vol. 117, no. 3, pp. 489-501, Dec. 2014.   DOI
3 K. Wang, G. Cao, D. Meng, W. Chen, and W. Cao, "Automatic fall detection of human in video using combination of features," in Proc. of 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Shenzhen, China, pp. 1228-1233, 2016.
4 Q. Feng, C. Gao, L. Wang, Y. Zhao, T. Song, and Q. Li, "Spatio-temporal fall event detection in complex scenes using attention guided LSTM," Pattern Recognition Letter, vol. 130, no. 1, pp. 242-249, Feb. 2020.   DOI
5 M. Chamle, K. G. Gunale and K. K. Warhade, "Automated unusual event detection in video surveillance," in Proc. of 2016 International Conference on Inventive Computation Technologies (ICICT), Coimbatore, India, pp. 1-4, 2016.
6 Y. Hsu, J. Perng and H. Liu, "Development of a vision based pedestrian fall detection system with back propagation neural network," in Proc. of 2015 IEEE/SICE International Symposium on System Integration (SII), Nagoya, Japan, pp. 433-437, 2015.
7 L. Yang, Y. Ren, H. Hu, and B. Tian, "New fast fall detection method based on spatio-temporal context tracking of head by using depth images," Sensors, vol. 15, no. 9, pp. 23004-23019, Sep. 2015.   DOI
8 M. Mubashir, L. Shao, and L. Seed, "A survey on fall detection: Principles and approaches," Neurocomputing, vol. 100, no.1, pp. 144-152, Jan. 2013.   DOI
9 G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, Jul. 2006.   DOI
10 T. Tamura, T. Yoshimura, M. Sekine, M. Uchida and O. Tanaka, "A Wearable Airbag to Prevent Fall Injuries," IEEE Transactions on Information Technology in Biomedicine, vol. 13, no. 6, pp. 910-914, Nov. 2009.   DOI
11 A. Irtaza, S. M. Adnan, S. Aziz, A. Javed, M. O. Ullah and M. T. Mahmood, "A framework for fall detection of elderly people by analyzing environmental sounds through acoustic local ternary patterns," in Proc. of 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada, pp. 1558-1563, 2017.
12 D. Anderson, J. M. Keller, M. Skubic, X. Chen and Z. He, "Recognizing Falls from Silhouettes," in Proc. of 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, New York, USA, pp. 6388-6391, 2006.
13 W. Min, L. Yao, Z. Lin, and L. Liu, ''Support vector machine approach to fall recognition based on simplified expression of human skeleton action and fast detection of start key frame using torso angle," IET Computer Vision, vol. 12, no. 8, pp. 1133-1140, Dec. 2018.   DOI
14 X. Cai, S. Li, X. Liu and G. Han, "A Novel Method Based on Optical Flow Combining with Wide Residual Network for Fall Detection," in Proc. of 2019 IEEE 19th International Conference on Communication Technology (ICCT), Xi'an, China, pp. 715-718, 2019.
15 S. Ji, W. Xu, M. Yang, and K. Yu, "3D convolutional neural networks for human action recognition," IEEE Transaction Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 221-231, Jan. 2013.   DOI
16 A. Poonsri and W. Chiracharit, "Improvement of fall detection using consecutive-frame voting," in Proc. of 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, Thailand, pp. 1-4, 2018.
17 C. Vishnu, R. Datla, D. Roy, S. Babu and C. K. Mohan, "Human Fall Detection in Surveillance Videos Using Fall Motion Vector Modeling," IEEE Sensors Journal, vol. 21, no. 15, pp. 17162-17170, Aug. 2021.   DOI
18 Y. Yun and I. Y. Gu, "Human fall detection via shape analysis on Riemannian manifolds with applications to elderly care," in Proc. of 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, Canada, 2015, pp. 3280-3284, 2015.
19 S. Li, X. Song, S. Xu, H. Qi and Y. Xue, "Dilated spatial-temporal convolutional auto-encoders for human fall detection in surveillance videos," ICT Express, July. 2022..
20 Y. Chen, W. Li, L. Wang, J. Hu, and M. Ye, "Vision-based fall event detection in complex background using attention guided bi-directional LSTM," IEEE Access, vol. 8, no.1, pp. 161337-161348, Sep. 2020.   DOI
21 A. Shahzad and K. Kim, "FallDroid: An Automated Smart-Phone-Based Fall Detection System Using Multiple Kernel Learning," IEEE Transactions on Industrial Informatics, vol. 15, no. 1, pp. 35-44, Jan. 2019.   DOI
22 W. Lie, A. T. Le and G. Lin, "Human fall-down event detection based on 2D skeletons and deep learning approach," in Proc. of 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, Thailand, pp. 1-4, 2018.
23 L. Ren and Y. Peng, "Research of Fall Detection and Fall Prevention Technologies: A Systematic Review," IEEE Access, vol. 7, no .1, pp. 77702-77722, Jun. 2019.   DOI
24 F. Harrou, N. Zerrouki, Y. Sun and A. Houacine, "Vision-based fall detection system for improving safety of elderly people," IEEE Instrumentation & Measurement Magazine, vol. 20, no. 6, pp. 49-55, Dec. 2017.   DOI
25 X. Xi, M. Tang, S. M. Miran, and Z. Luo, "Evaluation of feature extraction and recognition for activity monitoring and fall detection based on wearable sEMG sensors," Sensors, vol. 17, no. 6, pp. 1229-1249, May. 2017.   DOI
26 I. Charfi, J. Miteran, J. Dubois, M. Atri and R. Tourki, "Optimised spatio-temporal descriptors for real-time fall detection: Comparison of SVM and Adaboost based classification," Journal of Electronic Imaging, vol. 22, no. 4, pp. 041106-041106, Oct. 2013.   DOI
27 D. Droghini, E. Principi, S. Squartini, P. Olivetti, and F. Piazza, "Human fall detection by using an innovative floor acoustic sensor," in Multidisciplinary Approaches to Neural Computing, Smart Innovation, Systems and Technologies, vol. 69, Berlin, GER, 2018, pp. 97-107.
28 K. Chaccour, R. Darazi, A. Hajjam el Hassans and E. Andres, "Smart carpet using differential piezoresistive pressure sensors for elderly fall detection," in Proc. of 2015 IEEE 11th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), Abu Dhabi, United Arab Emirates, pp. 225-229, 2015.
29 Z. Bian, J. Hou, L. Chau and N. Magnenat-Thalmann, "Fall Detection Based on Body Part Tracking Using a Depth Camera," IEEE Journal of Biomedical and Health Informatics, vol. 19, no. 2, pp. 430-439, March 2015.   DOI
30 M. Yu, S. M. Naqvi and J. Chambers, "Fall detection in the elderly by head tracking," in Proc. of 2009 IEEE/SP 15th Workshop on Statistical Signal Processing, Cardiff, UK, pp. 357-360, 2009.
31 R. Cucchiara, A. Prati, R. Vezzani, "A multi-camera vision system for fall detection and alarm generation," Expert Systems, vol. 24, no. 5, pp. 334-345, Nov. 2007.   DOI
32 A. Marcos, G. Azkune, and I. Arganda-Carreras, "Vision-based fall detection with convolutional neural networks," Wireless Communication and Mobile Computing, vol. 2017, no. 1, pp. 1-16, Dec. 2017.
33 Z. Tu, W. Xie, Q. Qin, R. Poppe, R. C. Veltkamp, B. Li, and J. Yuan, "Multi-stream CNN: Learning representations based on human-related regions for action recognition," Pattern Recognition, vol. 79, no. 1, pp. 32-43, Jul. 2018.   DOI
34 N. Lu, Y. Wu, L. Feng, and J. Song, "Deep learning for fall detection: Three-dimensional CNN combined with LSTM on video kinematic data," IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 1, pp. 314-323, Jan. 2019.   DOI
35 Y. Wang, W. Zhou, Q. Zhang, and H. Li, "Enhanced action recognition with visual attribute-augmented 3D convolutional neural network," in Proc. of 2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), San Diego, CA, USA, pp. 1-4, 2018.