• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.21 no.3
• /
• pp.558-567
• /
• 2017

In this research, we developed an efficient real-time human exercise pose estimation system using multiple Kinects. The main objective of this system is to measure and recognize the user's posture (such as knee curl or lunge) more accurately by employing Kinects on the front and the sides. Especially it is designed as an extensible and modular method which enables to support various additional postures in the future. This system is configured as multiple clients and the Unity3D server. The client processes Kinect skeleton data and send to the server. The server performs the multiple-Kinect calibration process and then applies the pose estimation algorithm based on the Kinect-based posture recognition model using feature extractions and the weighted averaging of feature values for different Kinects. This paper presents the design and implementation of the human exercise pose estimation system using multiple Kinects and also describes how to build and execute an interactive Unity3D exergame.

• KIPS Transactions on Software and Data Engineering
• /
• v.10 no.5
• /
• pp.187-194
• /
• 2021

A person's behavioral recognition is the recognition of what a person does according to joint movements. To this end, we utilize computer vision tasks that are utilized in image processing. Human behavior recognition is a safety accident response service that combines deep learning and CCTV, and can be applied within the safety management site. Existing studies are relatively lacking in behavioral recognition studies through human joint keypoint extraction by utilizing deep learning. There were also problems that were difficult to manage workers continuously and systematically at safety management sites. In this paper, to address these problems, we propose a method to recognize risk behavior using only joint keypoints and joint motion information. AlphaPose, one of the pose estimation methods, was used to extract joint keypoints in the body part. The extracted joint keypoints were sequentially entered into the Long Short-Term Memory (LSTM) model to be learned with continuous data. After checking the behavioral recognition accuracy, it was confirmed that the accuracy of the "Lying Down" behavioral recognition results was high.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.9 no.8
• /
• pp.3136-3150
• /
• 2015

A vision-based 3D tracking of articulated human hand is one of the major issues in the applications of human computer interactions and understanding the control of robot hand. This paper presents an improved approach for tracking and recovering the 3D position and orientation of a human hand using the Kinect sensor. The basic idea of the proposed method is to solve an optimization problem that minimizes the discrepancy in 3D shape between an actual hand observed by Kinect and a hypothesized 3D hand model. Since each of the 3D hand pose has 23 degrees of freedom, the hand articulation tracking needs computational excessive burden in minimizing the 3D shape discrepancy between an observed hand and a 3D hand model. For this, we first created a 3D hand model which represents the hand with 17 different parts. Secondly, Random Forest classifier was trained on the synthetic depth images generated by animating the developed 3D hand model, which was then used for Haar-like feature-based classification rather than performing per-pixel classification. Classification results were used for estimating the joint positions for the hand skeleton. Through the experiment, we were able to prove that the proposed method showed improvement rates in hand part recognition and a performance of 20-30 fps. The results confirmed its practical use in classifying hand area and successfully tracked and recovered the 3D hand pose in a real time fashion.

• Journal of the Korea Society of Computer and Information
• /
• v.28 no.9
• /
• pp.17-25
• /
• 2023

In this paper, we propose a model that can perform human pose estimation through a MobileViT-based model with fewer parameters and faster estimation. The based model demonstrates lightweight performance through a structure that combines features of convolutional neural networks with features of Vision Transformer. Transformer, which is a major mechanism in this study, has become more influential as its based models perform better than convolutional neural network-based models in the field of computer vision. Similarly, in the field of human pose estimation, Vision Transformer-based ViTPose maintains the best performance in all human pose estimation benchmarks such as COCO, OCHuman, and MPII. However, because Vision Transformer has a heavy model structure with a large number of parameters and requires a relatively large amount of computation, it costs users a lot to train the model. Accordingly, the based model overcame the insufficient Inductive Bias calculation problem, which requires a large amount of computation by Vision Transformer, with Local Representation through a convolutional neural network structure. Finally, the proposed model obtained a mean average precision of 0.694 on the MS COCO benchmark with 3.28 GFLOPs and 9.72 million parameters, which are 1/5 and 1/9 the number compared to ViTPose, respectively.

• 제어로봇시스템학회:학술대회논문집
• /
• 2005.06a
• /
• pp.735-739
• /
• 2005

This paper presents a new system to estimate the head pose of human in interactive indoor environment that has dynamic illumination change and large working space. The main idea of this system is to suggest a new morphological feature for estimating head angle from stereo disparity map. When a disparity map is obtained from stereo camera, the matching confidence value can be derived by measurements of correlation of the stereo images. Applying a threshold to the confidence value, we also obtain the specific morphology of the disparity map. Therefore, we can obtain the morphological shape of disparity map. Through the analysis of this morphological property, the head pose can be estimated. It is simple and fast algorithm in comparison with other algorithm which apply facial template, 2D, 3D models and optical flow method. Our system can automatically segment and estimate head pose in a wide range of head motion without manual initialization like other optical flow system. As the result of experiments, we obtained the reliable head orientation data under the real-time performance.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.17 no.4
• /
• pp.191-197
• /
• 2022

This study proposes an image-based Pose Intention Network (PIN) algorithm for rehabilitation via patients' intentions. The purpose of the PIN algorithm is for enabling an active rehabilitation exercise, which is implemented by estimating the patient's motion and classifying the intention. Existing rehabilitation involves the inconvenience of attaching a sensor directly to the patient's skin. In addition, the rehabilitation device moves the patient, which is a passive rehabilitation method. Our algorithm consists of two steps. First, we estimate the user's joint position through the OpenPose algorithm, which is efficient in estimating 2D human pose in an image. Second, an intention classifier is constructed for classifying the motions into three categories, and a sequence of images including joint information is used as input. The intention network also learns correlations between joints and changes in joints over a short period of time, which can be easily used to determine the intention of the motion. To implement the proposed algorithm and conduct real-world experiments, we collected our own dataset, which is composed of videos of three classes. The network is trained using short segment clips of the video. Experimental results demonstrate that the proposed algorithm is effective for classifying intentions based on a short video clip.

• International conference on construction engineering and project management
• /
• 2022.06a
• /
• pp.839-846
• /
• 2022

Traditionally, the construction industry has shown low labor productivity and productivity growth. To improve labor productivity, it must first be accurately measured. The existing method uses work-sampling techniques through observation of workers' activities at certain time intervals on site. However, a disadvantage of this method is that the results may differ depending on the observer's judgment and may be inaccurate in the case of a large number of missed scenarios. Therefore, this study proposes a model to automate labor productivity measurement by monitoring workers' actions using a deep learning-based pose estimation method. The results are expected to contribute to productivity improvement on construction sites.

• Journal of the Korean Society of Industry Convergence
• /
• v.26 no.1
• /
• pp.113-119
• /
• 2023

In this paper, we propose an optimization of a pose estimation deep learning model for recognition of abnormal behavior in unmanned stores using radio frequencies. The radio frequency use millimeter wave in the 30 GHz to 300 GHz band. Due to the short wavelength and strong straightness, it is a frequency with less grayness and less interference due to radio absorption on the object. A millimeter wave radar is used to solve the problem of personal information infringement that may occur in conventional CCTV image-based pose estimation. Deep learning-based pose estimation models generally use convolution neural networks. The convolution neural network is a combination of convolution layers and pooling layers of different types, and there are many cases of convolution filter size, number, and convolution operations, and more cases of combining components. Therefore, it is difficult to find the structure and components of the optimal posture estimation model for input data. Compared with conventional millimeter wave-based posture estimation studies, it is possible to explore the structure and components of the optimal posture estimation model for input data using genetic algorithms, and the performance of optimizing the proposed posture estimation model is excellent. Data are collected for actual unmanned stores, and point cloud data and three-dimensional keypoint information of Kinect Azure are collected using millimeter wave radar for collapse and property damage occurring in unmanned stores. As a result of the experiment, it was confirmed that the error was moored compared to the conventional posture estimation model.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.11 no.7
• /
• pp.2579-2585
• /
• 2010

This paper proposes a dynamic human pose tracking method using motion-based search strategy from an image sequence obtained from a monocular camera. The proposed method compares the image features between 3D human model projections and real input images. The method repeats the process until predefined criteria and then estimates 3D human pose that generates the best match. When searching for the best matching configuration with respect to the input image, the search region is determined from the estimated 2D image motion and then search is performed randomly for the body configuration conducted within that search region. As the 2D image motion is highly constrained, this significantly reduces the dimensionality of the feasible space. This strategy have two advantages: the motion estimation leads to an efficient allocation of the search space, and the pose estimation method is adaptive to various kinds of motion.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.24 no.4
• /
• pp.607-616
• /
• 2014

Face recognition is a big challenge in surveillance system since different rotation angles of the face make the difficulty to recognize the face of the same person. This paper proposes a novel method to recognize face with different head poses by using 3D information of the face. Firstly, head pose estimation (estimation of different head pose angles) is accomplished by the POSIT algorithm. Then, 3D face image data is constructed by using head pose estimation. After that, 2D image and the constructed 3D face matching is performed. Face verification is accomplished by using commercial face recognition SDK. Performance evaluation of the proposed method indicates that the error range of head pose estimation is below 10 degree and the matching rate is about 95%.