• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.20 no.9
• /
• pp.525-531
• /
• 2019

Efforts to utilize human gestures to effectively implement a more natural and interactive interface between humans and computers have been ongoing in recent years. In this paper, we propose a new algorithm that accepts consecutive three-dimensional (3D) depth images, defines a hand model, and robustly extracts the human hand region based on six palm joints and 15 finger joints. Then, the 3D depth images are adaptively binarized to exclude non-interest areas, such as the background, and accurately extracts only the hand of the person, which is the area of interest. Experimental results show that the presented algorithm detects only the human hand region 2.4% more accurately than the existing method. The hand region extraction algorithm proposed in this paper is expected to be useful in various practical applications related to computer vision and image processing, such as gesture recognition, virtual reality implementation, 3D motion games, and sign recognition.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2021.07a
• /
• pp.15-16
• /
• 2021

본 논문에서는 RGB 영상 데이터를 입력으로 하여 mediapipe의 손 포즈 추정 알고리즘을 적용해 손가락 관절 및 주요 부위의 위치를 얻고 이를 기반으로 딥러닝 모델에 학습 후 손동작 인식 방법을 제안한다. 연속된 프레임에서 한 손의 손가락 주요 부위 간 좌표를 얻고 차분 벡터의 x, y좌표를 저장한 후 Conv1D, Bidirectional GRU, Transformer를 결합한 딥러닝 모델에 학습 후 손동작 인식 분류를 하였다. IC4You Gesture Dataset 의 한 손 동적 데이터 9개 클래스에 적용한 결과 99.63%의 손동작 인식 정확도를 얻었다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2018.10a
• /
• pp.686-689
• /
• 2018

3D 손 포즈 추정(Hand Pose Estimation, HPE)은 스마트 인간 컴퓨터 인터페이스를 위해서 중요한 기술이다. 이 연구에서는 딥러닝 방법을 기반으로 하여 단일 RGB-Depth 카메라로 촬영한 양손의 3D 손 자세를 실시간으로 인식하는 손 포즈 추정 시스템을 제시한다. 손 포즈 추정 시스템은 4단계로 구성된다. 첫째, Skin Detection 및 Depth cutting 알고리즘을 사용하여 양손을 RGB와 깊이 영상에서 감지하고 추출한다. 둘째, Convolutional Neural Network(CNN) Classifier는 오른손과 왼손을 구별하는데 사용된다. CNN Classifier 는 3개의 convolution layer와 2개의 Fully-Connected Layer로 구성되어 있으며, 추출된 깊이 영상을 입력으로 사용한다. 셋째, 학습된 CNN regressor는 추출된 왼쪽 및 오른쪽 손의 깊이 영상에서 손 관절을 추정하기 위해 다수의 Convolutional Layers, Pooling Layers, Fully Connected Layers로 구성된다. CNN classifier와 regressor는 22,000개 깊이 영상 데이터셋으로 학습된다. 마지막으로, 각 손의 3D 손 자세는 추정된 손 관절 정보로부터 재구성된다. 테스트 결과, CNN classifier는 오른쪽 손과 왼쪽 손을 96.9%의 정확도로 구별할 수 있으며, CNN regressor는 형균 8.48mm의 오차 범위로 3D 손 관절 정보를 추정할 수 있다. 본 연구에서 제안하는 손 포즈 추정 시스템은 가상 현실(virtual reality, VR), 증강 현실(Augmented Reality, AR) 및 융합 현실 (Mixed Reality, MR) 응용 프로그램을 포함한 다양한 응용 분야에서 사용할 수 있다.

• Journal of Broadcast Engineering
• /
• v.17 no.3
• /
• pp.447-458
• /
• 2012

We present a method to recognize hand gestures using skeletal joint data obtained from Microsoft's Kinect sensor. We propose a combination feature of multi-angle histograms robust to orientation variations to represent the observation sequence of skeletons. The proposed feature efficiently represents the orientation variations of gestures that can be occurred according to person or environment by combining the multiple angle histograms with various angular-quantization levels. The gesture represented as combination of multi-angle histograms and random decision forest classifier improve the recognition performance. We conduct the experiments in hand gesture dataset obtained from a kinect sensor and show that our method outperforms the other methods by comparing the recognition performance.

• Journal of Broadcast Engineering
• /
• v.18 no.3
• /
• pp.380-392
• /
• 2013

A method to classify a golf swing motion into 7 sections using a Kinect sensor and a fuzzy system is proposed. The inputs to the fuzzy logic are the positions of golf club and its head, which are extracted from the information of golfer's joint position and color information obtained by a Kinect sensor. The proposed method consists of three modules: one for extracting the joint's information, another for detecting and tracking of a golf club, and the other for classifying golf swing motions. The first module extracts the hand's position among the joint information provided by a Kinect sensor. The second module detects the golf club as well as its head with the Hough line transform based on the hand's coordinate. Using a fuzzy logic as a classification engine reduces recognition errors and, consequently, improves the performance of robust classification. From the experiments of real-time video clips, the proposed method shows the reliability of classification by 85.2%.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.24 no.12
• /
• pp.1574-1580
• /
• 2020

Traditional pose estimation methods include using special devices or images through image processing. The disadvantage of using a device is that the environment in which the device can be used is limited and costly. The use of cameras and image processing has the advantage of reducing environmental constraints and costs, but the performance is lower. CNN(Convolutional Neural Networks) were studied for pose estimation just using only camera without these disadvantage. Various techniques were proposed to increase cognitive performance. In this paper, the effect of the skip connection on the network was experimented by using various skip connections on the joint recognition of the hand. Experiments have confirmed that the presence of additional skip connections other than the basic skip connections has a better effect on performance, but the network with downward skip connections is the best performance.

• Journal of KIISE:Computing Practices and Letters
• /
• v.7 no.3
• /
• pp.248-259
• /
• 2001

This paper proposes a human action recognition scheme to recognize nonverbal human communications automatically. Based on the principle that a human body action can be defined as a combination of multiple articulation movements, we use the method of inferencing stochastic grammars to understand each human actions. We measure and quantize each human action in 3D world-coordinate, and make two sets of 4-chain-code for xy and zy projection plane. Based on the fact that the neighboring information among articulations is an essential element to distinguish actions, we designed a new stochastic inference procedure to apply the neighboring information of hands. Our proposed scheme shows better recognition rate than that of other general stochastic inference procedures. ures.

• Journal of KIISE:Software and Applications
• /
• v.35 no.12
• /
• pp.780-788
• /
• 2008

Modeling hand poses and tracking its movement are one of the challenging problems in computer vision. There are two typical approaches for the reconstruction of hand poses in 3D, depending on the number of cameras from which images are captured. One is to capture images from multiple cameras or a stereo camera. The other is to capture images from a single camera. The former approach is relatively limited, because of the environmental constraints for setting up multiple cameras. In this paper we propose a method of reconstructing 3D hand poses from a 2D input image sequence captured from a single camera by means of Belief Propagation in a graphical model and recognizing a finger clicking motion using a hidden Markov model. We define a graphical model with hidden nodes representing joints of a hand, and observable nodes with the features extracted from a 2D input image sequence. To track hand poses in 3D, we use a Belief Propagation algorithm, which provides a robust and unified framework for inference in a graphical model. From the estimated 3D hand pose we extract the information for each finger's motion, which is then fed into a hidden Markov model. To recognize natural finger actions, we consider the movements of all the fingers to recognize a single finger's action. We applied the proposed method to a virtual keypad system and the result showed a high recognition rate of 94.66% with 300 test data.

• Journal of the HCI Society of Korea
• /
• v.6 no.1
• /
• pp.1-8
• /
• 2011

As virtual realty has become an issue of providing immersive services, in the area of virtual realty, it has been actively investigated to develop user interfaces for immersive interaction. In this paper, we propose a gesture recognition based immersive user interface by using an IR LED embedded helmet and data gloves in order to reflect the user's movements to the virtual reality environments effectively. The system recognizes the user's head movements by using the IR LED embedded helmet and IR signal transmitter, and the hand gestures with the data gathered from data gloves. In case of hand gestures recognition, it is difficult to recognize accurately with the general recognition model because there are various hand gestures since human hands consist of many articulations and users have different hand sizes and hand movements. In this paper, we applied the Mixture-of-Experts based gesture recognition for various hand gestures of multiple users accurately. The movement of the user's head is used to change the perspection in the virtual environment matching to the movement in the real world, and the gesture of the user's hand can be used as inputs in the virtual environment. A head mounted display (HMD) can be used with the proposed system to make the user absorbed in the virtual environment. In order to evaluate the usefulness of the proposed interface, we developed an interface for the virtual orchestra environment. The experiment verified that the user can use the system easily and intuituvely with being entertained.

• Journal of Korea Game Society
• /
• v.12 no.5
• /
• pp.79-88
• /
• 2012

In this paper, we present a real-time hand pose recognition method to provide an intuitive user interface through hand poses or movements without a keyboard and a mouse. For this, the areas of right and left hands are segmented from the depth camera image, and noise removal is performed. Then, the rotation angle and the centroid point of each hand area are calculated. Subsequently, a circle is expanded at regular intervals from a centroid point of the hand to detect joint points and end points of the finger by obtaining the midway points of the hand boundary crossing. Lastly, the matching between the hand information calculated previously and the hand model of previous frame is performed, and the hand model is recognized to update the hand model for the next frame. This method enables users to predict the hidden fingers through the hand model information of the previous frame using temporal coherence in consecutive frames. As a result of the experiment on various hand poses with the hidden fingers using both hands, the accuracy showed over 95% and the performance indicated over 32 fps. The proposed method can be used as a contactless input interface in presentation, advertisement, education, and game applications.