• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.9 no.8
• /
• pp.3136-3150
• /
• 2015

A vision-based 3D tracking of articulated human hand is one of the major issues in the applications of human computer interactions and understanding the control of robot hand. This paper presents an improved approach for tracking and recovering the 3D position and orientation of a human hand using the Kinect sensor. The basic idea of the proposed method is to solve an optimization problem that minimizes the discrepancy in 3D shape between an actual hand observed by Kinect and a hypothesized 3D hand model. Since each of the 3D hand pose has 23 degrees of freedom, the hand articulation tracking needs computational excessive burden in minimizing the 3D shape discrepancy between an observed hand and a 3D hand model. For this, we first created a 3D hand model which represents the hand with 17 different parts. Secondly, Random Forest classifier was trained on the synthetic depth images generated by animating the developed 3D hand model, which was then used for Haar-like feature-based classification rather than performing per-pixel classification. Classification results were used for estimating the joint positions for the hand skeleton. Through the experiment, we were able to prove that the proposed method showed improvement rates in hand part recognition and a performance of 20-30 fps. The results confirmed its practical use in classifying hand area and successfully tracked and recovered the 3D hand pose in a real time fashion.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.10
• /
• pp.1323-1329
• /
• 2021

For hand expression recognition, hand pose recognition based on the static shape of the hand and hand gesture recognition based on the dynamic hand movement are used together. In this paper, we propose a hierarchical hand pose model based on finger position and shape for hand expression recognition. For hand pose recognition, a finger model representing the finger state and a hand pose model using the finger state are hierarchically constructed, which is based on the open source MediaPipe. The finger model is also hierarchically constructed using the bending of one finger and the touch of two fingers. The proposed model can be used for various applications of transmitting information through hands, and its usefulness was verified by applying it to number recognition in sign language. The proposed model is expected to have various applications in the user interface of computers other than sign language recognition.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2021.01a
• /
• pp.277-278
• /
• 2021

최근 손동작 인식에 대한 연구들이 활발하다. 하지만 대부분 Depth 정보를 포함한3D 정보를 필요로 한다. 이는 기존 연구들이 Depth 카메라 없이는 동작하지 않는다는 한계점이 있다는 것을 의미한다. 본 프로젝트는 Depth 카메라를 사용하지 않고 2D 이미지에서 Hand Keypoint Detection을 통해 손동작 인식을 하는 방법론을 제안한다. 학습 데이터 셋으로 Facebook에서 제공하는 InterHand2.6M 데이터셋[1]을 사용한다. 제안 방법은 크게 두 단계로 진행된다. 첫째로, Object Detection으로 Hand Detection을 수행한다. 데이터 셋이 어두운 배경에서 촬영되어 실 사용 환경에서 Detection 성능이 나오지 않는 점을 해결하기 위한 이미지 합성 Augmentation 기법을 제안한다. 둘째로, Keypoint Detection으로 21개의 Hand Keypoint들을 얻는다. 실험을 통해 유의미한 벡터들을 생성한 뒤 Annoy (Approximate nearest neighbors Oh Yeah) Tree를 생성한다. 생성된 Annoy Tree들로 후처리 작업을 거친 뒤 최종 Pose Estimation을 완료한다. Annoy Tree를 사용한 Pose Estimation에서는 NN(Neural Network)을 사용한 것보다 빠르며 동등한 성능을 냈다.

• The Journal of Korea Robotics Society
• /
• v.10 no.4
• /
• pp.223-229
• /
• 2015

This paper proposes a novel method for detection of hand raising poses from images acquired from a single camera attached to a mobile robot that navigates unknown dynamic environments. Due to unconstrained illumination, a high level of variance in human appearances and unpredictable backgrounds, detecting hand raising gestures from an image acquired from a camera attached to a mobile robot is very challenging. The proposed method first detects faces to determine the region of interest (ROI), and in this ROI, we detect hands by using a HOG-based hand detector. By using the color distribution of the face region, we evaluate each candidate in the detected hand region. To deal with cases of failure in face detection, we also use a HOG-based hand raising pose detector. Unlike other hand raising pose detector systems, we evaluate our algorithm with images acquired from the camera and images obtained from the Internet that contain unknown backgrounds and unconstrained illumination. The level of variance in hand raising poses in these images is very high. Our experiment results show that the proposed method robustly detects hand raising poses in complex backgrounds and unknown lighting conditions.

• Journal of Computing Science and Engineering
• /
• v.10 no.3
• /
• pp.95-101
• /
• 2016

In this study, we propose a new system for constructing parallel corpora for sign languages, which are generally under-resourced in comparison to spoken languages. In order to achieve scalability and accessibility regarding data collection and corpus construction, our system utilizes deep learning-based techniques and predicts depth information to perform pose estimation on hand information obtainable from video recordings by a single RGB camera. These estimated poses are then transcribed into expressions in SignWriting. We evaluate the accuracy of hand tracking and hand pose estimation modules of our system quantitatively, using the American Sign Language Image Dataset and the American Sign Language Lexicon Video Dataset. The evaluation results show that our transcription system has a high potential to be successfully employed in constructing a sizable sign language corpus using various types of video resources.

• Journal of the Korea Society of Computer and Information
• /
• v.17 no.7
• /
• pp.155-166
• /
• 2012

This paper proposes a new method of estimating the hand pose through the Mean-Shift tracking algorithm using the fusion of color and depth information for marker-less augmented reality. On marker-less augmented reality, the most of previous studies detect the hand region using the skin color from simple experimental background. Because finger features should be detected on the hand, the hand pose that can be measured from cameras is restricted considerably. However, the proposed method can easily detect the hand pose from complex background through the new Mean-Shift tracking method using the fusion of the color and depth information from 3D sensor. The proposed method of estimating the hand pose uses the gravity point and two random points on the hand without largely constraints. The proposed Mean-Shift tracking method has about 50 pixels error less than general tracking method just using color value. The augmented reality experiment of the proposed method shows results of its performance being as good as marker based one on the complex background.

• The Journal of the Korea Contents Association
• /
• v.5 no.2
• /
• pp.267-274
• /
• 2005

In this paper, we proposed an algorithm which cognize hand pose in real time using only image. Hand recognizes using edge orientation histogram which comes under a constant quantity of 2D appearance because hand pose is intricate. This method suit hand pose recognition in real time because it extracts hand space accurately, has little computation quantify, and is less sensitive to lighting change using color information in complicated background. Method which reduces recognition error using principal component analysis method to can recognize through hand shape presentation direction change is explained. A case that hand shape changes by turning 3D also by using this method is possible to recognize. Besides, principal component space creation time is reduced remarkably because edge directional data is used.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2009.01a
• /
• pp.341-345
• /
• 2009

We present an augmented reality (AR) application for cell phone where users put a virtual pet on their palms and play/interact with the pet by moving their hands and fingers naturally. The application is fundamentally based on hand/palm pose recognition and finger motion estimation, which is the main concern in this paper. We propose a fast and efficient hand/palm pose recognition method which uses natural features (e.g. direction, width, contour shape of hand region) extracted from a hand image with prior knowledge for hand shape or geometry (e.g. its approximated shape when a palm is open, length ratio between palm width and pal height). We also propose a natural interaction method which recognizes natural motion of fingers such as opening/closing palm based on fingertip tracking. Based on the proposed methods, we developed and tested the AR application on an ultra-mobile PC (UMPC).

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.12
• /
• pp.1770-1776
• /
• 2021

For hand expression recognition, hand pose recognition based on the static shape of the hand and hand gesture recognition based on hand movement are used together. In this paper, we proposed a hand expression recognition method that recognizes symbols based on the trajectory of a hand movement on a virtual blackboard. In order to recognize a sign drawn by hand on a virtual blackboard, not only a method of recognizing a sign from a hand movement, but also hand pose recognition for finding the start and end of data input is also required. In this paper, MediaPipe was used to recognize hand pose, and LSTM(Long Short Term Memory), a type of recurrent neural network, was used to recognize hand gesture from time series data. To verify the effectiveness of the proposed method, it was applied to the recognition of numbers written on a virtual blackboard, and a recognition rate of about 94% was obtained.

• Journal of Broadcast Engineering
• /
• v.19 no.6
• /
• pp.942-956
• /
• 2014

This paper proposes a real-time hand pose tracking method using a monocular RGB camera. Hand tracking has high ambiguity since a hand has a number of degrees of freedom. Thus, to reduce the ambiguity the proposed method adopts the step-by-step estimation scheme: a palm pose estimation, a finger yaw motion estimation, and a finger pitch motion estimation, which are performed in consecutive order. Assuming a hand to be a plane, the proposed method utilizes a planar hand model, which facilitates a hand model regeneration. The hand model regeneration modifies the hand model to fit a current user's hand, and improves robustness and accuracy of the tracking results. The proposed method can work in real-time and does not require GPU-based processing. Thus, it can be applied to various platforms including mobile devices such as Google Glass. The effectiveness and performance of the proposed method will be verified through various experiments.