• Journal of Institute of Control, Robotics and Systems
• /
• v.18 no.1
• /
• pp.54-61
• /
• 2012

In this paper, the pose estimation method for the satellite GTB (Ground Test Bed) using vision/MEMS IMU (Inertial Measurement Unit) integrated system is presented. The GTB for verifying a satellite system on the ground is similar to the mobile robot having thrusters and a reaction wheel as actuators and floating on the floor by compressed air. The EKF (Extended Kalman Filter) is also used for fusion of MEMS IMU and vision system that consists of a single camera and infrared LEDs that is ceiling landmarks. The fusion filter generally utilizes the position of feature points from the image as measurement. However, this method can cause position error due to the bias of MEMS IMU when the camera image is not obtained if the bias is not properly estimated through the filter. Therefore, it is proposed that the fusion method which uses the position of feature points and the velocity of the camera determined from optical flow of feature points. It is verified by experiments that the performance of the proposed method is robust to the bias of IMU compared to the method that uses only the position of feature points.

• Smart Media Journal
• /
• v.6 no.3
• /
• pp.49-56
• /
• 2017

Images and Videos that include the human face contain a lot of information. Therefore, accurately extracting human face is a very important issue in the field of computer vision. However, in real life, human faces have various shapes and textures. To adapt to these variations, A model-based approach is one of the best ways in which unknown data can be represented by the model in which it is built. However, the model-based approach has its weaknesses when the motion between two frames is big, it can be either a sudden change of pose or moving with fast speed. In this paper, we propose an enhanced human face-tracking model. This approach included human face detection and motion estimation using Cascaded Convolutional Neural Networks, and continuous human face tracking and modeling correction steps using the Active Appearance Model. A proposed system detects human face in the first input frame and initializes the models. On later frames, Cascaded CNN face detection is used to estimate the target motion such as location or pose before applying the old model and fit new target.

• The Journal of Korea Robotics Society
• /
• v.17 no.3
• /
• pp.373-380
• /
• 2022

This paper considers a method of fast correspondence matching for iterative closest point (ICP) algorithm. In robotics, the ICP algorithm and its variants have been widely used for pose estimation by finding the translation and rotation that best align two point clouds. In computational perspectives, the main difficulty is to find the correspondence point on the reference point cloud to each observed point. Jump-table-based correspondence matching is one of the methods for reducing computation time. This paper proposes a method that corrects errors in an existing jump-table-based correspondence matching algorithm. The criterion activating the use of jump-table is modified so that the correspondence matching can be applied to the situations, such as point-cloud registration problems with highly curved surfaces, for which the existing correspondence-matching method is non-applicable. For demonstration, both hardware and simulation experiments are performed. In a hardware experiment using Hokuyo-10LX LiDAR sensor, our new algorithm shows 100% correspondence matching accuracy and 88% decrease in computation time. Using the F1TENTH simulator, the proposed algorithm is tested for an autonomous driving scenario with 2D range-bearing point cloud data and also shows 100% correspondence matching accuracy.

• International journal of advanced smart convergence
• /
• v.11 no.2
• /
• pp.128-134
• /
• 2022

Nowadays, people consume a lot of content such as web dramas or K-pop videos through mobile devices such as smartphones, and the market for indirect advertisements through these web dramas or K-pop videos is also increasing every year. In order to lead to the immediate purchase of indirect products in web dramas, a system that allows consumers to purchase immediately at the time the products appear in the drama is needed. In this paper, we propose a system to allow viewers to purchase products worn by celebrities immediately when viewers see and click on them. When a user clicks on a video, it recognizes the product worn by the celebrity, and displays information on the screen on the most similar product corresponding to the recognized product, allowing them to go to the seller's site where they can purchase it. In order for such a system to operate stably, a pose estimation and siamese network-based system is proposed. The proposed system will primarily be released as a streaming service in the form of an app or web page that connects the products in web dramas or other K-pop video contents screened on the mobile with e-commerce. Furthermore, in the future, the technology is expected to be used globally in various industries such as smart mobility and display kiosks.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2020.11a
• /
• pp.68-71
• /
• 2020

최근 딥러닝을 이용한 인체 자세 추정(human pose estimation) 연구가 활발히 진행되고 있다. 그 중 구조가 간단하면서도 성능이 강력하여 널리 사용되고 있는 딥러닝 네트워크 모델은 이미지 분류(image classification)에 사용되는 백본 네트워크(backbone network)와 디컨볼루션 출력망(deconvolution head network)을 이어 붙인 구조를 갖는다[1]. 기존의 디컨볼루션 출력망은 디컨볼루션 층을 쌓아 낮은 해상도의 특징맵을 모두 높은 해상도로 변환한 후 최종 인체 자세 추정을 하는데 이는 다양한 해상도에서 얻어낸 특징들을 골고루 활용하기 힘들다는 단점이 있다. 따라서 본 논문에서는 매 디컨볼루션 층 이후에 인체 자세 추정을 하여 다양한 해상도에서 연산을 하고 이를 종합하여 최종 인체 자세 추정을 하는 방법을 제안한다. 실험 결과 Res50 과 기존의 디컨볼루션 출력망의 경우 0.717 AP 를 얻었는데 Res101 과 기존의 디컨볼루션 출력망을 사용한 결과 50% 이상의 파라미터 수 증가와 함께 0.727 AP, 즉 0.010AP 의 성능 향상이 이루어졌다. 이에 반해 Res50 에 다중 해상도 디컨볼루션 출력망을 사용한 결과 약 1%의 파라미터 수 증가 만으로 0.720 AP, 즉 0.003 AP 의 성능 향상이 이루어졌다. 이를 통해 디컨볼루션 출력망 구조를 개선하면 매우 적은 파라미터 수 증가 만으로도 인체 자세 추정의 성능을 효과적으로 향상시킬 수 있음을 확인하였다.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.54 no.2
• /
• pp.106-114
• /
• 2017

In this paper, we propose the method of designing an extended kalman filter in order to accurately measure the position of the spatial-phase system using sensor fusion. We use the quaternion as a state variable in expressing the attitude of an object. Then, the attitude of rigid body can be calculated from the accelerometer and magnetometer by applying the Gauss-Newton method. We estimate the changes of state by using the measurements obtained from the gyroscope, the quaternion, and the vision informations by ARVR_SDK. To increase the accuracy of estimation, we designed and implemented the extended kalman filter, which showed excellent ability to adjust and compensate the sensor error. As a result, we could experimentally demonstrate that the reliability of the attitude estimation value can be significantly increased.

• Journal of Broadcast Engineering
• /
• v.27 no.3
• /
• pp.341-350
• /
• 2022

In this paper, we propose a deep learning based on-device augmented reality (AR) system in which multiple input images are used to implement the correct occlusion in a real environment. The proposed system is composed of three technical steps; camera pose estimation, depth estimation, and object augmentation. Each step employs various mobile frameworks to optimize the processing on the on-device environment. Firstly, in the camera pose estimation stage, the massive computation involved in feature extraction is parallelized using OpenCL which is the GPU parallelization framework. Next, in depth estimation, monocular and multiple image-based depth image inference is accelerated using the mobile deep learning framework, i.e. TensorFlow Lite. Finally, object augmentation and occlusion handling are performed on the OpenGL ES mobile graphics framework. The proposed augmented reality system is implemented as an application in the Android environment. We evaluate the performance of the proposed system in terms of augmentation accuracy and the processing time in the mobile as well as PC environments.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.24 no.3
• /
• pp.147-156
• /
• 2024

This paper introduces the design of a turtle neck posture correction and prevention system for users of digital devices for a long time. The number of forward head posture patients in Korea increased by 13% from 2018 to 2021, and has not yet improved according to the latest statistics at the present time. Because of the nature of the disease, prevention is more important than treatment. Therefore, in this paper, we designed a system based on built-camera in most laptops to increase the accessiblility of the system, and utilize the features such as Pose Estimation, Face Landmarks Detection, Iris Tracking, and Depth Estimation of Google Mediapipe to prevent the need to produce artificial intelligence models and allow users to easily prevent forward head posture.

• The Journal of Korea Robotics Society
• /
• v.10 no.3
• /
• pp.171-177
• /
• 2015

A telerobot offers a more engaging and enjoyable interaction with people at a distance by communicating via audio, video, expressive gestures, body pose and proxemics. To provide its potential benefits at a reasonable cost, this paper presents a telepresence robot system for video communication which can deliver speaker's head motion through its display stanchion. Head gestures such as nodding and head-shaking can give crucial information during conversation. We also can assume a speaker's eye-gaze, which is known as one of the key non-verbal signals for interaction, from his/her head pose. In order to develop an efficient head tracking method, a 3D cylinder-like head model is employed and the Harris corner detector is combined with the Lucas-Kanade optical flow that is known to be suitable for extracting 3D motion information of the model. Especially, a skin color-based face detection algorithm is proposed to achieve robust performance upon variant directions while maintaining reasonable computational cost. The performance of the proposed head tracking algorithm is verified through the experiments using BU's standard data sets. A design of robot platform is also described as well as the design of supporting systems such as video transmission and robot control interfaces.

• Journal of the Korean Society of Manufacturing Technology Engineers
• /
• v.26 no.2
• /
• pp.223-229
• /
• 2017

Recently, robotic bin-picking tasks have drawn considerable attention, because flexibility is required in robotic assembly tasks. Generally, stereo camera systems have been used widely for robotic bin-picking, but these have two limitations: First, computational burden for solving correspondence problem on stereo images increases calculation time. Second, errors in image processing and camera calibration reduce accuracy. Moreover, the errors in robot kinematic parameters directly affect robot gripping. In this paper, we propose a method of correcting the bin-picking error by using trinocular vision system which consists of two stereo cameras andone hand-eye camera. First, the two stereo cameras, with wide viewing angle, measure object's pose roughly. Then, the 3rd hand-eye camera approaches the object, and corrects the previous measurement of the stereo camera system. Experimental results show usefulness of the proposed method.