Journal of Sensor Science and Technology (센서학회지)

The Korean Sensors Society (한국센서학회)
권호 표지
DOI QR코드

DOI QR Code

CNN-based Fall Detection Model for Humanoid Robots

CNN 기반의 인간형 로봇의 낙상 판별 모델

  • Shin-Woo Park (Department of Robot and Smart System Engineering, Kyungpook National University) ;
  • Hyun-Min Joe (Department of Robot and Smart System Engineering, Department of Artificial Intelligence, Kyungpook National University)
  • 박신우 (경북대학교 로봇 및 스마트시스템공학과) ;
  • 조현민 (경북대학교 로봇 및 스마트시스템공학과, 인공지능학과)
  • Received : 2023.12.28
  • Accepted : 2024.01.05
  • Published : 2024.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

Humanoid robots, designed to interact in human environments, require stable mobility to ensure safety. When a humanoid robot falls, it causes damage, breakdown, and potential harm to the robot. Therefore, fall detection is critical to preventing the robot from falling. Prevention of falling of a humanoid robot requires an operator controlling a crane. For efficient and safe walking control experiments, a system that can replace a crane operator is needed. To replace such a crane operator, it is essential to detect the falling conditions of humanoid robots. In this study, we propose falling detection methods using Convolution Neural Network (CNN) model. The image data of a humanoid robot are collected from various angles and environments. A large amount of data is collected by dividing video data into frames per second, and data augmentation techniques are used. The effectiveness of the proposed CNN model is verified by the experiments with the humanoid robot MAX-E1.

Keywords

Acknowledgement

본 논문은 정부(과학기술정보통신부)의 재원으로 한국 연구 재단의 지원을 받아 수행된 연구임 (No.2022R1C1C101311512).

References

  1. S. Kajita, H. Hirukawa, K. Harada, and K. Yokoi, Introduction to humanoid robotics, Springer Berlin, Heidelberg, pp. 1-210, 2014.
  2. H. M. Joe and J. H. Oh, "A robust balance-control framework for the terrain-blind bipedal walking of a humanoid robot on unknown and uneven terrain", Sensors, Vol. 19, No. 19, pp. 4194(1)-4194(19), 2019.
  3. H. M. Joe and J. H. Oh, "Balance recovery through model predictive control based on capture point dynamics for biped walking robot", Rob. Auton. Syst., Vol. 105, pp. 1-10, 2018.
  4. H. M. Joe, "Locomotion control of a humanoid robot using a biped walking pattern based on physical interaction", Journal of Institute of Control, Robotics and Systems, Vol. 27, No. 9, pp.668-675, 2021.
  5. K. Ogata, K. Terada, and Y. Kuniyoshi, "Falling motion control for humanoid robots while walking", Proc. of 2007 7th IEEE-RAS International Conference on Humanoid Robots, pp. 306-311, Pittsburgh, USA, 2007.
  6. K. Ogata, K. Terada, and Y. Kuniyoshi, "Real-time selection and generation of fall damage reduction actions for humanoid robots", Proc. of 2008 8th IEEE-RAS International Conference on Humanoid Robots, pp. 232-238, Daejeon, Korea (South), 2008.
  7. S. W. Park and H. M. Joe, "Mobile crane for path tracking of humanoid robots", Journal of Institute of Control, Robotics and Systems, Vol. 29, No. 9, pp. 734-739, 2023.
  8. R. Subburaman, D. Kanoulas, N. Tsagarakis, and J. Lee, "A survey on control of humanoid fall over", Rob. Auton. Syst., Vol. 166, p. 104443, 2023.
  9. X. Xinjilefu, S. Feng, and C. G. Atkeson, "Center of mass estimator for humanoids and its application in modelling error compensation, fall detection and prevention", Proc. of 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 67-73, Seoul, Korea (South), 2015.
  10. R. Subburaman, D. Kanoulas, L. Muratore, N. G. Tsagarakis, and J. Lee, "Human inspired fall prediction method for humanoid robots", Rob. Auton. Syst., Vol. 121, 2019.
  11. R. Subburaman, J. Lee, D. G. Caldwell, and N. G. Tsagarakis, "Multi-Sensor based fall prediction method for humanoid robots", Proc. of 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), pp. 102-108, Baden-Baden, Germany, 2016.
  12. M. Arfaq, R. S. Dewanto, and D. Pramadihanto, "Fall Detection in T-FLoW Humanoid Robot: V-REP Simulation", Proc. of 2018 International Electronics Symposium on Engineering Technology and Applications (IES-ETA), pp. 224-228, Bali, Indonesia, 2018.
  13. T. Wu, Z. Yu, X. Chen, C. Dong, Z. Gao, and Q. Huang, "Falling prediction based on machine learning for biped robots", J. Intell. Robot. Syst., Vol. 103, No. 68, pp. 1-14, 2021.
  14. D. Gouaillier, V. Hugel, P. Blazevic, C. Kilner, J. Monceaux, P. Lafourcade, B. Marnier, J. Serre, and B. Maisonnier, "Mechatronic design of NAO humanoid", Proc. of 2009 IEEE international conference on robotics and automation, pp. 769-774, Kobe, Japan, 2009.
  15. C. Zampino, F. Biancospino, M. Brienza, F. Laus, G. Di Stefano, R. Romano, A. Pennisi, V. Suriani, and D. D. Bloisi, "Fall detection using NAO robot pose estimation in RoboCup SPL matches", Proc. of the 9th Italian Workshop on Artificial Intelligence and Robotics, pp. 1-8, Udine, Italy, 2022.
  16. D. Osokin, "Real-time 2d multi-person pose estimation on cpu: Lightweight openpose", arXiv preprint arXiv:1811. 12004, pp. 1-5, 2018.
  17. K. O'Shea and R. Nash, "An introduction to convolutional neural networks", arXiv preprint arXiv:1511.08458, pp. 1-11, 2015.
  18. S. W. Park, S. J. Ha, and H. M. Joe, "Fall detection of humanoid robot using CNN", Proc. of 2023 The 38th ICROS Annual Conference (ICROS 2023), pp. 892-893, Samcheok, Korea (South), 2023.
  19. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks", Commun. ACM, Vol. 60, No. 6, pp. 84-90, 2017.