Browse > Article
http://dx.doi.org/10.9708/jksci.2021.26.04.039

A Study on Sensor-Based Upper Full-Body Motion Tracking on HoloLens  

Park, Sung-Jun (Company of DataReality)
Publication Information
Journal of the Korea Society of Computer and Information / v.26, no.4, 2021 , pp. 39-46 More about this Journal
Abstract
In this paper, we propose a method for the motion recognition method required in the industrial field in mixed reality. In industrial sites, movements (grasping, lifting, and carrying) are required throughout the upper full-body, from trunk movements to arm movements. In this paper, we use a method composed of sensors and wearable devices that are not vision-based such as Kinect without using heavy motion capture equipment. We used two IMU sensors for the trunk and shoulder movement, and used Myo arm band for the arm movements. Real-time data coming from a total of 4 are fused to enable motion recognition for the entire upper body area. As an experimental method, a sensor was attached to the actual clothes, and objects were manipulated through synchronization. As a result, the method using the synchronization method has no errors in large and small operations. Finally, through the performance evaluation, the average result was 50 frames for single-handed operation on the HoloLens and 60 frames for both-handed operation.
Keywords
Mixed Reality; HoloLens; Myo; IMU; Upper Full-Body;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Hui Liang, Junsong Yuan, Daniel Thalmann, Nadia Magnenat Thalmann, "AR in Hand : Egocentric Palm Pose Tracking and Gesture Recognition for Augmented Reality Applications," ACM MM Proceedings, pp. 743-744, Oct.2015, DOI:10.1145/2733373.2807972   DOI
2 Yizheng Gu, Chun Yu, Zhipeng Li, Weiqi Li, Shuchang Xu, Xiaoying Wei, Yuanchun Shi, "Accurate and Low-Latency Sensing of Touch Contact on Any Surface with Finger-Worn IMU Sensor," UIST ACM Proceedings, pp. 1059-1070, Oct.2019,DOI:10.1145/3332165.3347947   DOI
3 Taylor Frantz, Bart Jansen, Johnny Duerinck, Jef Vandemeuleb roucke, "Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation," Healthc Technol Left, Vol. 5, No. 5, pp. 221-225, Oct. 2018, DOI :10.1049/htl.2018.5079   DOI
4 Lucio Tommaso De Paolis, Valerio De Luca, "The impact of the input interface in a virtual environment: Vive controller and the Myo armband," Springer Vritual Reality, Vol. 24, pp. 483-502, Nov. 2019, DOI : 10.1007/s10055-019-00409-6   DOI
5 Carol Naranjo-Valero, Sriram Srinivasa, Achim Ebert, and Hamann, "Enhanced Freehand Interaction by Combining Vision and EMG-based systems in Mixed Reality Environments," CSCE
6 Rafael Radkowski, "Hololens Integration into a Multi-Kinect Tracking Environment," IEEE ISMAR-Adjunct, Apr. 2019, DOI:10.1109/ISMAR-Adjunct.2018.00052   DOI
7 Maximilian Spicher, Brian D.Hall, Michael Nebeling, "What is Mixed Reality?," CHI Conference, No.537, pp. 1-15, May. 2019, DOI:10.1145/3290605.3300767   DOI
8 Markus Funk, Mareike Kritzler, Florian Michahelles, "Hololens is more than air Tap: natural and intuitive interaction with holograms," IoT 17 Proceedings, No.31, pp. 1-2, Oct. 2017, DOI:10.1145/3131542.3140267   DOI
9 Mathi, Krish Chaithanya, "Augment HoloLens' Body Recognition and Tracking Capabilities Using Kinect," Wright State University, Computer Science, 2016, http://orcid.org/0000-0002-8185-8142
10 Poonsiri Jailungka, Siam Charoenseang, "Intuitive 3D Model Prototyping with Leap Motion and Microsoft HoloLens," Springer LNCS, Vol.10903, pp. 269-284, Jun. 2018, DOI:10.1007/978-3-319-91250-9_21   DOI
11 Tobias Mulling, Mithileysh Sathiyanarayanan, "Characteristics of Hand Gesture Navigation: a case study using a wearable device(MYO)," British HCI'15, pp. 283-284, Jul. 2015, DOI:10.1145/2783446.2783612   DOI
12 Patrick Hubner, Kate Clintworth, Qingyi Liu, Martin Weinmann and Sven Wursthorn,"Evaluation of HoloLens Tracking and Depth Sensing for Indoor Mapping Applications," Sensors, Vol.20, No.4, pp. 283-284, Feb. 2020, DOI:10.3390/s20041021   DOI
13 Mathias Unberath, Kevin Yu, Roghayeh Barmaki, Alex Johnson, Nassir Navab, "Augment Yourself: Mixed Reality Self-Augmen tation Using Optical See-through Head-mounted Displays and Physical Mirrors," IEEE VR TVCG, Nov. 2017, arXiv:2007.02884[cs.HC]
14 Juan David Hincapie-Ramos, Kasim Ozacar, Pourang P.Irani, Yoshifumi Kitamura, "GyroWand: IMU-based Raycasting for Augmented Reality Head-Mounted Displays," ACM SUI Proceedings, pp. 89-98, Aug. 2015, DOI:10.1145/2788940.2788947   DOI