• Journal of Korea Multimedia Society
• /
• v.25 no.9
• /
• pp.1257-1265
• /
• 2022

Recognition of sign language is challenging due to the occlusion of hands, accuracy of hand gestures, and high computational costs. In recent years, deep learning techniques have made significant advances in this field. Although these methods are larger and more complex, they cannot manage long-term sequential data and lack the ability to capture useful information through efficient information processing with faster convergence. In order to overcome these challenges, we propose a word-level sign language recognition (SLR) system that combines a real-time human pose detection library with the minimized version of the gated recurrent unit (GRU) model. Each gate unit is optimized by discarding the depth-weighted reset gate in GRU cells and considering only current input. Furthermore, we use sigmoid rather than hyperbolic tangent activation in standard GRUs due to performance loss associated with the former in deeper networks. Experimental results demonstrate that our pose-based optimized GRU (Pose-OGRU) outperforms the standard GRU model in terms of prediction accuracy, convergency, and information processing capability.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.17 no.4
• /
• pp.191-197
• /
• 2022

This study proposes an image-based Pose Intention Network (PIN) algorithm for rehabilitation via patients' intentions. The purpose of the PIN algorithm is for enabling an active rehabilitation exercise, which is implemented by estimating the patient's motion and classifying the intention. Existing rehabilitation involves the inconvenience of attaching a sensor directly to the patient's skin. In addition, the rehabilitation device moves the patient, which is a passive rehabilitation method. Our algorithm consists of two steps. First, we estimate the user's joint position through the OpenPose algorithm, which is efficient in estimating 2D human pose in an image. Second, an intention classifier is constructed for classifying the motions into three categories, and a sequence of images including joint information is used as input. The intention network also learns correlations between joints and changes in joints over a short period of time, which can be easily used to determine the intention of the motion. To implement the proposed algorithm and conduct real-world experiments, we collected our own dataset, which is composed of videos of three classes. The network is trained using short segment clips of the video. Experimental results demonstrate that the proposed algorithm is effective for classifying intentions based on a short video clip.

• The Journal of Korea Robotics Society
• /
• v.14 no.2
• /
• pp.104-113
• /
• 2019

Object manipulation in cluttered environments remains an open hard problem. In cluttered environments, grasping objects often fails for various reasons. This paper proposes a novel task and motion planning scheme to grasp objects obstructed by other objects in cluttered environments. Task and motion planning (TAMP) aims to generate a sequence of task-level actions where its feasibility is verified in the motion space. The proposed scheme contains an open-loop consisting of three distinct phases: 1) Generation of a task-level skeleton plan with pose references, 2) Instantiation of pose references by motion-level search, and 3) Re-planning task based on the updated state description. By conducting experiments with simulated robots, we show the high efficiency of our scheme.

• Journal of Korea Multimedia Society
• /
• v.24 no.2
• /
• pp.336-343
• /
• 2021

AI technology is spreading not only to industrial fields, but also to culture, art, and content fields. In this paper, we proposed a system based on AI technology that can automate the process of reproducing contents using characters for movie contents. After creating the basic appearance of the character by using the StyleGAN2 model from the video extracted from the movie contents, analyzing the character's personality and propensity using the extracted dialogue data, it was determined from the contemplative appearance based on the yin-yang and five elements to the character's propensity. Accordingly, the external characteristics are reflected in the character. Using the OpenPose model, a character's motion is created, and the finally generated data is integrated to reproduce the content. It is expected that many movie contents can be reproduced through the study of the proposed system.

• International conference on construction engineering and project management
• /
• 2022.06a
• /
• pp.361-369
• /
• 2022

Unsafe actions and behaviors of workers cause most accidents at construction sites. Nowadays, occupational safety is a top priority at construction sites. However, this problem often requires money and effort from investors or construction owners. Therefore, decreasing the accidents rates of workers and saving monitoring costs for contractors is necessary at construction sites. This study proposes an unsafe behavior detection method based on a skeleton model to classify three common unsafe behaviors on the scaffold: climbing, jumping, and running. First, the OpenPose method is used to obtain the workers' key points. Second, all skeleton datasets are aggregated from the temporary size. Third, the key point dataset becomes the input of the action classification model. The method is effective, with an accuracy rate of 89.6% precision and 90.5% recall of unsafe actions correctly detected in the experiment.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2020.11a
• /
• pp.1056-1059
• /
• 2020

최근 드론과 1인 미디어 시장의 성장으로, 영상 촬영 분야에서의 드론 산업이 활발하게 발전되고 있다. 본 논문에서는 딥러닝 기반 다중 객체 인식 기술인 Openpose를 활용하여 인물촬영을 위한 음성인식 드론 제어 시스템을 제안한다. 해당 시스템은 자연어 처리된 음성명령어를 통해 드론이 각 촬영 객체에 대한 회전, 초점변화 등 실제 영상촬영기법에 사용되는 다수의 동작을 수행할 수 있도록 한다. 최종적으로 96.2%의 정확도로 음성명령에 따라 동작을 수행하는 것을 확인할 수 있다. 이는 누구나 전문적 지식이나 경험 없이 음성만으로 쉽게 드론을 제어할 수 있을 것으로 기대된다.

• Communications of the Korean Mathematical Society
• /
• v.11 no.4
• /
• pp.1067-1075
• /
• 1996

We show that every ultrapure space is weakly star reducible, and that every countably compact weakly star reducible space is compact. We also pose open problems.

• Journal of Korea Multimedia Society
• /
• v.22 no.5
• /
• pp.535-546
• /
• 2019

Recently, there have been a lot of researches on the whole face replacement system, but it is not easy to obtain stable results due to various attitudes, angles and facial diversity. To produce a natural synthesis result when replacing the face shown in the video image, technologies such as face area detection, feature extraction, face alignment, face area segmentation, 3D attitude adjustment and facial transposition should all operate at a precise level. And each technology must be able to be interdependently combined. The results of our analysis show that the difficulty of implementing the technology and contribution to the system in facial replacement technology has increased in facial feature point extraction and facial alignment technology. On the other hand, the difficulty of the facial transposition technique and the three-dimensional posture adjustment technique were low, but showed the need for development. In this paper, we propose four facial replacement models such as 2-D Faceswap, OpenPose, Deekfake, and Cycle GAN, which are suitable for the Cox platform. These models have the following features; i.e. these models include a suitable model for front face pose image conversion, face pose image with active body movement, and face movement with right and left side by 15 degrees, Generative Adversarial Network.

• Korean Journal of Computational Design and Engineering
• /
• v.16 no.4
• /
• pp.285-291
• /
• 2011

A pose estimation process from medical images is calculating locations and orientations of objects obtained from Computed Tomography (CT) volume data utilizing X-ray images from two directions. In this process, digitally reconstructed radiograph (DRR) images of spatially transformed objects are generated and compared to X-ray images repeatedly until reasonable transformation matrices of the objects are found. The DRR generation and image comparison take majority of the total time for this pose estimation. In this paper, a fast DRR generation technique based on GPU parallel computing is introduced. A volume ray-casting algorithm is explained with brief vector operations and a parallelization technique of the algorithm using Compute Unified Device Architecture (CUDA) is discussed. This paper also presents the implementation results and time measurements comparing to those from pure-CPU implementation and open source toolkit.

• Journal of the Society of Disaster Information
• /
• v.19 no.3
• /
• pp.498-509
• /
• 2023

Purpose: This paper proposes a method detecting the falling of a maintenance worker in the underground utility tunnel, by applying deep learning techniques using CCTV video, and evaluates the applicability of the proposed method to the worker monitoring of the utility tunnel. Method: Each rule was designed to detect the falling of a maintenance worker by using the inference results from pre-trained YOLOv5 and OpenPose models, respectively. The rules were then integrally applied to detect worker falls within the utility tunnel. Result: Although the worker presence and falling were detected by the proposed model, the inference results were dependent on both the distance between the worker and CCTV and the falling direction of the worker. Additionally, the falling detection system using YOLOv5 shows superior performance, due to its lower dependence on distance and fall direction, compared to the OpenPose-based. Consequently, results from the fall detection using the integrated dual deep learning model were dependent on the YOLOv5 detection performance. Conclusion: The proposed hybrid model shows detecting an abnormal worker in the utility tunnel but the improvement of the model was meaningless compared to the single model based YOLOv5 due to severe differences in detection performance between each deep learning model