• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.2
• /
• pp.800-816
• /
• 2018

In recent years, video surveillance research has been able to recognize various behaviors of pedestrians and analyze the overall situation of objects by combining image analysis technology and deep learning method. Human Activity Recognition (HAR), which is important issue in video surveillance research, is a field to detect abnormal behavior of pedestrians in CCTV environment. In order to recognize human behavior, it is necessary to detect the human in the image and to estimate the pose from the detected human. In this paper, we propose a novel approach for 2D Human Pose Estimation based on object detection using RGB-D information. By adding depth information to the RGB information that has some limitation in detecting object due to lack of topological information, we can improve the detecting accuracy. Subsequently, the rescaled region of the detected object is applied to ConVol.utional Pose Machines (CPM) which is a sequential prediction structure based on ConVol.utional Neural Network. We utilize CPM to generate belief maps to predict the positions of keypoint representing human body parts and to estimate human pose by detecting 14 key body points. From the experimental results, we can prove that the proposed method detects target objects robustly in occlusion. It is also possible to perform 2D human pose estimation by providing an accurately detected region as an input of the CPM. As for the future work, we will estimate the 3D human pose by mapping the 2D coordinate information on the body part onto the 3D space. Consequently, we can provide useful human behavior information in the research of HAR.

• Journal of the Korea Society of Computer and Information
• /
• v.23 no.8
• /
• pp.9-16
• /
• 2018

In this paper, a kidnapping detection scheme in which human pose estimation is used to classify accurately between kidnapping cases and normal ones is proposed. To estimate human poses from input video, human's 10 joint information is extracted by OpenPose library. In addition to the features which are used in the previous study to represent the size change rates and the regularities of human activities, the human pose estimation features which are computed from the location of detected human's joints are used as the features to distinguish kidnapping situations from the normal accompanying ones. A frame-based kidnapping detection scheme is generated according to the selection of J48 decision tree model from the comparison of several representative classification models. When a video has more frames of kidnapping situation than the threshold ratio after two people meet in the video, the proposed scheme detects and notifies the occurrence of kidnapping event. To check the feasibility of the proposed scheme, the detection accuracy of our newly proposed scheme is compared with that of the previous scheme. According to the experiment results, the proposed scheme could detect kidnapping situations more 4.73% correctly than the previous scheme.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.6
• /
• pp.3182-3198
• /
• 2019

Vision-based Human Pose Estimation has been considered as one of challenging research subjects due to problems including confounding background clutter, diversity of human appearances and illumination changes in scenes. To tackle these problems, we propose to use a new multi-stage convolution machine for estimating human pose. To provide better heatmap prediction of body joints, the proposed machine repeatedly produces multiple predictions according to stages with receptive field large enough for learning the long-range spatial relationship. And stages are composed of various modules according to their strategic purposes. Pyramid stacking module and dilation module are used to handle problem of human pose at multiple scales. Their multi-scale information from different receptive fields are fused with concatenation, which can catch more contextual information from different features. And spatial and channel information of a given input are converted to gating factors by squeezing the feature maps to a single numeric value based on its importance in order to give each of the network channels different weights. Compared with other ConvNet-based architectures, we demonstrated that our proposed architecture achieved higher accuracy on experiments using standard benchmarks of LSP and MPII pose datasets.

• KIPS Transactions on Software and Data Engineering
• /
• v.11 no.10
• /
• pp.419-426
• /
• 2022

In this paper, we propose a method to improve the accuracy of 3D human pose estimation model in various move motions. Existing human pose estimation models have some problems of jitter, inversion, swap, miss that cause miss coordinates when estimating human poses. These problems cause low accuracy of pose estimation models to detect exact coordinates of human poses. We propose a method that consists of detection and correction methods to handle with these problems. Deep learning-based outlier detection method detects outlier of human pose coordinates in move motion effectively and rule-based correction method corrects the outlier according to a simple rule. We have shown that the proposed method is effective in various motions with the experiments using 2D golf swing motion data and have shown the possibility of expansion from 2D to 3D coordinates.

• International conference on construction engineering and project management
• /
• 2022.06a
• /
• pp.328-335
• /
• 2022

With the advance of robot capabilities and functionalities, construction robots assisting construction workers have been increasingly deployed on construction sites to improve safety, efficiency and productivity. For close-proximity human-robot collaboration in construction sites, robots need to be aware of the context, especially construction worker's behavior, in real-time to avoid collision with workers. To recognize human behavior, most previous studies obtained 3D human poses using a single camera or an RGB-depth (RGB-D) camera. However, single-camera detection has limitations such as occlusions, detection failure, and sensor malfunction, and an RGB-D camera may suffer from interference from lighting conditions and surface material. To address these issues, this study proposes a novel method of 3D human pose estimation by extracting 2D location of each joint from multiple images captured at the same time from different viewpoints, fusing each joint's 2D locations, and estimating the 3D joint location. For higher accuracy, the probabilistic representation is used to extract the 2D location of the joints, considering each joint location extracted from images as a noisy partial observation. Then, this study estimates the 3D human pose by fusing the probabilistic 2D joint locations to maximize the likelihood. The proposed method was evaluated in both simulation and laboratory settings, and the results demonstrated the accuracy of estimation and the feasibility in practice. This study contributes to ensuring human safety in close-proximity human-robot collaboration by providing a novel method of 3D human pose estimation.

• Journal of Broadcast Engineering
• /
• v.27 no.5
• /
• pp.718-727
• /
• 2022

The Korean domestic cartoon industry has grown explosively by 65% compared to the previous year. Then the market size is expected to exceed KRW 1 trillion. However, excessive work results in health deterioration. Moreover, this working environment makes the production of human resources insufficient, repeating a vicious cycle. Although some tasks require creation activity during cartoon production, there are still a lot of simple repetitive tasks. Therefore, this study aimed to develop a method for creating a character pose through human pose estimation (HPE). The HPE is to detect key points for each joint of a user. The primary role of the proposed method was to make each joint of the character match that of the human. The proposed method enabled us to create the pose of the two-dimensional cartoon character through the results. Furthermore, it was possible to save the static image for one character pose and the video for continuous character pose.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.19 no.1
• /
• pp.9-17
• /
• 2024

It is widely recognized that for 3D human pose estimation (HPE), dataset acquisition is expensive and the effectiveness of augmentation techniques of conventional visual recognition tasks is limited. We address these difficulties by presenting a simple but effective method that augments input images in terms of viewpoints when training a 3D human pose estimation (HPE) model. Our intuition is that meaningful variants of the input images for HPE could be obtained by viewing a human instance in the images from an arbitrary viewpoint different from that in the original images. The core idea is to synthesize new images that have self-occlusion and thus are difficult to predict at different viewpoints even with the same pose of the original example. We incorporate this idea into the training procedure of the 3D HPE model as an augmentation stage of the input samples. We show that a strategy for augmenting the synthesized example should be carefully designed in terms of the frequency of performing the augmentation and the selection of viewpoints for synthesizing the samples. To this end, we propose a new metric to measure the prediction difficulty of input images for 3D HPE in terms of the distance between corresponding keypoints on both sides of a human body. Extensive exploration of the space of augmentation probability choices and example selection according to the proposed distance metric leads to a performance gain of up to 6.2% on Human3.6M, the well-known pose estimation dataset.

• Smart Media Journal
• /
• v.13 no.4
• /
• pp.16-22
• /
• 2024

Pitching is a major part of baseball, so much so that it can be said to be the beginning of baseball. Analysis of accurate pitching motions is very important in terms of performance improvement and injury prevention. When analyzing the correct pitching motion, the currently used motion capture method has several critical environmental drawbacks. In this paper, we propose analysis of pitching motion using the RGB-based Human Pose Estimation (HPE) model to replace motion capture, which has these shortcomings, and use motion capture data and HPE data to verify its reliability. The similarity of the two data was verified by comparing joint coordinates using the Dynamic Time Warping (DTW) algorithm.

• Journal of information and communication convergence engineering
• /
• v.21 no.4
• /
• pp.337-345
• /
• 2023

In this study, we formulated a method that evaluates Taekwondo Poomsae performance using a series of choreographed training movements. Despite recent achievements in 3D human pose estimation (HPE) performance, the analysis of human actions remains challenging. In particular, Taekwondo Poomsae action analysis is challenging owing to the absence of time synchronization data and necessity to compare postures, rather than directly relying on joint locations owing to differences in human shapes. To address these challenges, we first decomposed human joint representation into joint rotation (posture) and limb length (body shape), then synchronized a comparison between test and reference pose sequences using DTW (dynamic time warping), and finally compared pose angles for each joint. Experimental results demonstrate that our method successfully synchronizes test action sequences with the reference sequence and reflects a considerable gap in performance between practitioners and professionals. Thus, our method can detect incorrect poses and help practitioners improve accuracy, balance, and speed of movement.

• KIPS Transactions on Software and Data Engineering
• /
• v.11 no.12
• /
• pp.517-524
• /
• 2022

Human pose estimation is a method of estimating a posture by extracting a human joint key point. When occlusion occurs, the joint key point extraction performance is lowered because the human joint is covered. The occlusion phenomenon is largely divided into three types of actions: self-contained, covered by other objects, and covered by background. In this paper, we propose an effective posture estimation method using a masking phenomenon enhancement technique. Although the posture estimation method has been continuously studied, research on the occlusion phenomenon of the posture estimation method is relatively insufficient. To solve this problem, the author proposes a data augmentation technique that intentionally masks human joints. The experimental results in this paper show that the intentional use of the blocking phenomenon enhancement technique is strong against the blocking phenomenon and the performance is increased.