The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Smart Wrist Band Considering Wrist Skin Curvature Variation for Real-Time Hand Gesture Recognition

실시간 손 제스처 인식을 위하여 손목 피부 표면의 높낮이 변화를 고려한 스마트 손목 밴드

  • Yun Kang (Department of Electro-Mechanical Systems Engineering, Korea University) ;
  • Joono Cheong (Department of Electro-Mechanical Systems Engineering, Korea University)
  • Received : 2022.09.13
  • Accepted : 2022.11.11
  • Published : 2023.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

This study introduces a smart wrist band system with pressure measurements using wrist skin curvature variation due to finger motion. It is easy to wear and take off without pre-adaptation or surgery to use. By analyzing the depth variation of wrist skin curvature during each finger motion, we elaborated the most suitable location of each Force Sensitive Resistor (FSR) to be attached in the wristband with anatomical consideration. A 3D depth camera was used to investigate distinctive wrist locations, responsible for the anatomically de-coupled thumb, index, and middle finger, where the variations of wrist skin curvature appear independently. Then sensors within the wristband were attached correspondingly to measure the pressure change of those points and eventually the finger motion. The smart wrist band was validated for its practicality through two demonstrative applications, i.e., one for a real-time control of prosthetic robot hands and the other for natural human-computer interfacing. And hopefully other futuristic human-related applications would be benefited from the proposed smart wrist band system.

Keywords

Acknowledgement

This results was supported by "Regional Innovation Strategy (RIS)" through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (MOE) (2021RIS-004)

References

  1. N. Parajuli, N. Sreenivasan, P. Bifulco, M. Cesarelli, S. Sa vino, V. Niola, D. Esposito, T. J. Hamilton, G. R. Naik, U. Gunawardana, and G. D. Gargiulo, "Real-time EMG based pattern recognition control for hand prostheses: a review on existing methods, challenges and future implementation," Sensors, vol. 19, no. 20, pp. 4596, 2019, DOI: 10.3390/ s19204596. 
  2. R. Gielissen, C. E. Dutilh, and J. J. Graafland, "Perceptions of price fairness: An empirical research," Business and Society, vol. 47, no. 3, pp. 370-89, 2007, DOI: 10.1177/ 0007650308316937. 
  3. J. T. Kate, S. Gerwin, and P. Breedveld, "3D-printed upper limb prostheses: a review," Disability and Rehabilitation: Assistive Technology, vol. 12, no. 3, pp. 300-314, 2017, DOI: 10.1080/17483107.2016.1253117. 
  4. S. M. Khan, A. A. Khan, and O. Farooq, "Selection of Fea tures and Classifiers for EMG-EEG-Based Upper Limb Assistive Devices - A Review," IEEE Reviews in Biomedical Engineering, vol. 13, pp. 248-260, 2019, DOI: 10.1109/ RBME.2019.2950897. 
  5. C. Wu, A. Song, Y. Ling, N. Wang, and L. Tian, "A control strategy with tactile perception feedback for EMG prosthetic hand," Journal of Sensors, 2015, DOI: 10.1155/2015/869175. 
  6. P. Polygerinos, K. C. Galloway, E. Savage, M. Herman, K. O. Donnell, and C. J. Walsh, "Soft robotic glove for hand rehabilitation and task specific training," 2015 IEEE International Conference on Robotics and Automation, Seattle, WA, USA, 2015, DOI: 10.1109/ ICRA.2015.7139597.
  7. F. J. Andres, A. Perez-Gonzalez, C. Rubert, J. Fuentes, and B. Sospedra, "Comparison of Grasping Performance of Tendon and Linkage Transmission Systems in an Electric-Powered Low-Cost Hand Prosthesis," Mechanisms Robotics, vol. 11, no. 1, pp. 14, DOI: 10.1115/1.4040491. 
  8. Y. Kang, "Motion recognition of human body using matrix pressure sensor and human body motion prediction system," Korea Patent 1021969620000, Mar., 5, 2020, DOI: 10.8080/1020200027488. 
  9. Y. Kang, "Control system of robotic articulated joint movement according to pressure sensor's continuative changing value by the tendon," Korea Patent 1018191580000, Nov., 12, 2016, DOI: 10.8080/1020160173692. 
  10. A. Dementyev and J. A. Paradiso, "Wrist Flex: low - power gesture input with wrist-worn pressure sensors," The 27th Annual ACM Symposium on User Interface Software and Technology, pp. 161-166, 2014, DOI: 10.1145/ 2642918.2647396. 
  11. D. Esposito, E. Andreozzi, G. D. Gargiulo, A. Fratini, G. D' Addio, G. R. Naik, and P. Bifulco, "A piezoresistive array armband with reduced number of sensors for hand gesture recognition," Frontiers in Neurorobotics, vol. 13, pp. 114, 2020, DOI: 10.3389/fnbot.2019.00114. 
  12. X. Chu, J. Liu, and S. Shimamoto, "A sensor - based hand gesture recognition system for Japanese sign language," 2021 IEEE Life Science and Technologies, Nara, Japan, 2021, DOI: 10.1109/LifeTech52111.2021.9391981. 
  13. Y. Zhang, B. Liu, and B. Liu, "Recognizing hand gestures with pressure - sensor - based motion sensing," IEEE Transactions on Biomedical Circuits and Systems, vol. 13, no. 6, pp. 1425-1436, 2019, DOI: 10.1109/ TBCAS.2019.2940030. 
  14. X. Liang, H. Li, W. Wang, Y. Liu, R. Ghannam, F. Fioran elli, and H. Heidari, "Fusion of wearable and contactless sensors for intelligent gesture recognition," Advanced Intelligent Systems, vol. 1, no. 7, 2019, DOI: 10.1002/ aisy.201900088. 
  15. P. B. Shull, S. Jiang, Y. Zhu, and X. Zhu, " Hand gesture recognition and finger angle estimation via wrist - worn modified barometric pressure sensing," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 4, pp. 724-732, 2019, DOI:10.1109/ TNSRE.2019.2905658.
  16. W. Wang, X. Liang, M. Assaad, and H. Heidari,"Wearable Wristworn Gesture Recognition Using Echo State Network," 2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS), Genoa, Italy, 2019, DOI: 10.1109/ICECS46596.2019.8965219. 
  17. M. Schuenke, E. Schulte, and U. Schumacher, " THIEME Atlas of Anatomy General Anatomy and Musculoskeletal System," NY: Stuttgart, 2006, [Online], https://books.google.co.kr/books?id=NK9TgTaGt6UC&lpg=PP1&pg=PP1#v=onepage&q&f=false, Accessed: Sept. 13, 2022. 
  18. D. Ragheb, A. Stanley, A. Gentili, T. Hughes, and C. B. C hung, "MR imaging of the finger tendons: normal anatomy and commonly encountered pathology," European Journal of Radiology, vol. 56, no. 3, pp. 296-306, 2005, DOI: 10.1016/j.ejrad.2005.03.011. 
  19. P. J. Keir and R. P. Wells, "Changes in geometry of the finger flexor tendons in the carpal tunnel with wrist posture and tendon load: an MRI study on normal wrists," Clinical Biomechanics, vol. 14, no. 9, pp. 635-645, 1999, DOI: 10.1016/ S0268-0033(99)00012-1. 
  20. Interlink Electronics, FSR Series Data Sheet, [Online], https://cdn2.hubspot.net/hubfs/3899023/Interlinkelectronics%20November2017/Docs/Datasheet_FSR.pdf, Accessed: Sept. 13, 2022. 
  21. C. Rossi, S. Savino, V. Niola, and S. Troncone, "A study of a robotic hand with tendon driven fingers," Robotica, vol. 33, no. 5, pp. 1034-1048, 2015, DOI: 10.1017/S0263574714001179. 
  22. Y. Cho, J. Cheong, and D. Kim, "Generalized Graph Representation of Tendon Driven Robot Mechanism," The Journal of Korea Robotics Society, vol. 9, no. 3, pp.178-184, Sept., 2014, DOI: 10.7746/jkros.2014.9.3.178. 
  23. G. Langevin, "InMoov open - sourced 3D printed life-size robot," [Online],http://inmoov.fr/hand-and-forarm/, Accessed: Sept. 13, 2022. 
  24. Y. Kang and J. Cheong, Demonstration1: Prosthetic Hand Control using Pressure Data Wrist Band, [Online], https://youtu.be/GZqww-zuocA, Accessed: Sept. 13, 2022. 
  25. Y. Kang and J. Cheong, Demonstration2: Display Control using Pressure Data Wrist Band, [Online], https://youtu.be/P67NYqg6wic, Accessed: Sept. 13, 2022.