Browse > Article
http://dx.doi.org/10.7746/jkros.2018.13.1.045

3D Convolutional Neural Networks based Fall Detection with Thermal Camera  

Kim, Dae-Eon (Ujin Technology)
Jeon, BongKyu (Robotics Program, KAIST(Korea Advanced Institute of Science and Technology))
Kwon, Dong-Soo (Department of Mechanical Engineering, KAIST(Korea Advanced Institute of Science and Technology))
Publication Information
The Journal of Korea Robotics Society / v.13, no.1, 2018 , pp. 45-54 More about this Journal
Abstract
This paper presents a vision-based fall detection system to automatically monitor and detect people's fall accidents, particularly those of elderly people or patients. For video analysis, the system should be able to extract both spatial and temporal features so that the model captures appearance and motion information simultaneously. Our approach is based on 3-dimensional convolutional neural networks, which can learn spatiotemporal features. In addition, we adopts a thermal camera in order to handle several issues regarding usability, day and night surveillance and privacy concerns. We design a pan-tilt camera with two actuators to extend the range of view. Performance is evaluated on our thermal dataset: TCL Fall Detection Dataset. The proposed model achieves 90.2% average clip accuracy which is better than other approaches.
Keywords
3D Convolutional neural networks; Fall detection; Thermal images;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. L. Hendrich, P. S. Bender, and A. Nyhuis, "Validation of the Hendrich II Fall Risk Model: A large concurrent case/control study of hospitalized patients," Applied Nursing Research, vol. 16, no. 1, pp. 9-21, February, 2003.   DOI
2 A. Kalache, D. Fu, S. Yoshida, W. Al-Faisal, L. Beattie, W. Chodzko-Zajko, H. Fu, K. James, S. Kalula, B. Krishnaswamy, N. Kronfol, P. Marin, I. Pike, D. Rose, V. Scott, J. Stevens, C. Todd, and G. Usha, "World Health Organisation Global Report on Falls Prevention in Older Age," Geneva: WHO 2007, World Health Organization, Department of Ageing & Life Course, ed., 2007.
3 L. Z. Rubenstein, "Falls in older people: epidemiology, risk factors and strategies for prevention," Age and Ageing, vol. 35, no. suppl_2, pp. ii37-ii41, 2006.
4 M. Mubashir, L. Shao, and L. Seed, "A survey on fall detection: Principles and approaches," Neurocomputing, vol. 100, no. 16, pp. 144-152, 2013.   DOI
5 J. G. Shin, "Prevalence and management of falls among elderly in a hospital," Unpublished master's thesis, Hanyang University, pp. 1-63, 2011.
6 H. Rimminen, J. Lindstrom, M. Linnavuo, and R. Sepponen, "Detection of falls among the elderly by a floor sensor using the electric near field," IEEE Transactions on Information Technology in Biomedicine, vol. 14, no. 6, pp. 1475-1476, 2010.   DOI
7 G. Mastorakis and D. Makris, "Fall detection system using Kinect's infrared sensor," Journal of Real-Time Image Processing, vol. 9, no. 4, pp. 635-646, December, 2014.   DOI
8 S. Vadivelu, S. Ganesan, O. V. R. Murthy, and A. Dhall, "Thermal Imaging Based Elderly Fall Detection," Computer Vision - ACCV 2016 Workshops, Taipei, Taiwan, pp. 541-553, 2016.
9 P. Scovanner, S. Ali, and M. Shah, "A 3-dimensional sift descriptor and its application to action recognition," 15th ACM international conference on Multimedia, Augsburg, Germany, pp. 357-360, 2007.
10 A. Klaeser, M. Marszalek, and C. Schmid, "A spatio-temporal descriptor based on 3d-gradients," British Machine Conference, pp. 99.1-99.10, 2008.
11 H. Wang and C. Schmid, "Action recognition with improved trajectories," 2013 IEEE International Conference on Computer Vision, Sydney, Australia, 2013, doi: 10.1109/ICCV.2013.441.   DOI
12 S. Ji, W. Xu, M. Yang, and Kai Yu, "3D convolutional neural networks for human action recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 221-231, Jan., 2013.   DOI
13 D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3d convolutional networks," 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 2015, doi: 10.1109/ICCV.2015.510.   DOI
14 K. Simonyan and A. Zisserman. "Two-stream convolutional networks for action recognition in videos," Advances in Neural Information Processing Systems 27, Montreal, Canada, pp. 568-576, 2014.
15 P. Molchanov, S. Gupta, K. Kim, and J. Kautz, "Hand gesture recognition with 3D convolutional neural networks," 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA, 2015, doi: 10.1109/CVPRW.2015.7301342.   DOI
16 M. A. Habib, M. S. Mohktar, S. B. Kamaruzzaman, K. S. Lim, T. M. Pin, and F. Ibrahim, "Smartphone-Based Solutions for Fall Detection and Prevention: Challenges and Open Issues," Sensors 2014, vol. 14, no. 4, pp. 7181-7208, 2014.   DOI
17 S. Shan and T. Yuan, "A wearable pre-impact fall detector using feature selection and support vector machine," IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING, Beijing, China, 2010, doi: 10.1109/ICOSP.2010.5656840.   DOI
18 B. Kaluza and M. Lustrek, "Fall detection and activity recognition methods for the confidence project: a survey," 10th International Multiconference, Ljubljana, Slovenia, pp. 22-25, 2008.
19 R. Igual, C. Medrano, and I. Plaza, "Challenges, issues and trends in fall detection systems," BioMedical Engineering, [Online], vol. 12, no. 1, pp. 66, DOI. Available: https://doi.org/10.1186/1475-925X-12-66.   DOI
20 M. K. Mulcahy and S. Kurkovsky. "Automatic Fall Detection using Mobile Devices," 2015 12th International Conference on Information Technology, Las Vegas, NV, USA, 2015, doi: 10.1109/ITNG.2015.98.   DOI
21 Z. Fu, E. Culurciello, P. Lichtsteiner, and Tobi Delbruck, "Fall detection using an address-event temporal contrast vision sensor," 2008 IEEE International Symposium on Circuits and Systems, Seattle, WA, USA, 2008, doi: 10.1109/ISCAS.2008.4541445.   DOI
22 S.-G. Miaou , P.-H. Sung, a nd C .-Y. Huang, "A c ustomized human fall detection system using omni-camera images and personal information," 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, Arlington, VA, USA, 2006, doi: 10.1109/DDHH.2006.1624792.   DOI
23 W. K. Wong, H. L. Lim, C. K. Loo, and W. S. Lim, "Home alone faint detection surveillance system using thermal camera," 2010 Second International Conference on Computer Research and Development, Kuala Lumpur, Malaysia, 2010, doi: 10.1109/ICCRD.2010.163.   DOI
24 S. Kido, T. Miyasaka, T. Tanaka, T. Shimizu, and T. Saga, "Fall detection in toilet rooms using thermal imaging sensors," 2009 IEEE/SICE International Symposium on System Integration, Tokyo, Japan, 2009, doi: 10.1109/SI.2009.5384550.   DOI
25 M. Baccouche,F. Mamalet, C. Wolf, C. Garcia, and A. Baskurt, "Sequential deep learning for human action recognition," International Workshop on Human Behavior Understanding, 2011, doi: 10.1007/978-3-642-25446-8_4.   DOI
26 K. Schindler and L. van Gool, "Action snippets: How many frames does human action recognition require?," 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008, doi: 10.1109/CVPR.2008.4587730.   DOI
27 A. Sixsmith, N. Johnson, and R. Whatmore, "Pyroelectric IR sensor arrays for fall detection in the older population," Journal de Physique IV, Vol. 128, pp. 153-160, September, 2005.
28 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, June, 2012.
29 L. van der Maaten and G. Hinton, "Visualizing data using t-SNE," Journal of Machine Learning Research 9, pp. 2579-2605, Nov., 2008.