• The KIPS Transactions:PartB
• /
• v.14B no.4
• /
• pp.311-320
• /
• 2007

This paper presents vision-based 3D facial expression animation technique and system which provide the robust 3D head pose estimation and real-time facial expression control. Many researches of 3D face animation have been done for the facial expression control itself rather than focusing on 3D head motion tracking. However, the head motion tracking is one of critical issues to be solved for developing realistic facial animation. In this research, we developed an integrated animation system that includes 3D head motion tracking and facial expression control at the same time. The proposed system consists of three major phases: face detection, 3D head motion tracking, and facial expression control. For face detection, with the non-parametric HT skin color model and template matching, we can detect the facial region efficiently from video frame. For 3D head motion tracking, we exploit the cylindrical head model that is projected to the initial head motion template. Given an initial reference template of the face image and the corresponding head motion, the cylindrical head model is created and the foil head motion is traced based on the optical flow method. For the facial expression cloning we utilize the feature-based method, The major facial feature points are detected by the geometry of information of the face with template matching and traced by optical flow. Since the locations of varying feature points are composed of head motion and facial expression information, the animation parameters which describe the variation of the facial features are acquired from geometrically transformed frontal head pose image. Finally, the facial expression cloning is done by two fitting process. The control points of the 3D model are varied applying the animation parameters to the face model, and the non-feature points around the control points are changed by use of Radial Basis Function(RBF). From the experiment, we can prove that the developed vision-based animation system can create realistic facial animation with robust head pose estimation and facial variation from input video image.

• Electronics and Telecommunications Trends
• /
• v.29 no.1
• /
• pp.11-20
• /
• 2014

동작인식(gesture recognition) 기술은 입력 영상으로부터 영상에 포함된 사람들의 동작을 인식하는 기술로써 영상감시(visual surveillance), 사람-컴퓨터 상호작용(human-computer interaction), 지능로봇(intelligence robot) 등 다양한 적용분야를 가진다. 특히 최근에는 저비용의 거리 센서(range sensor) 및 효율적인 3차원 자세 추정(3D pose estimation)기술의 등장으로 동작인식은 기존의 어려움들을 극복하고 다양한 산업분야에 적용이 가능할 정도로 발전을 거듭하고 있다. 본고에서는 그러한 거리영상(range image) 기반의 동작인식 기술에 대한 최신 연구동향을 살펴본다.

• Journal of the Korea Computer Graphics Society
• /
• v.20 no.2
• /
• pp.1-11
• /
• 2014

The quality of 2D-to-3D conversion depends on the accuracy of the assigned depth to scene objects. Manual depth painting for given objects is labor intensive as each frame is painted. Specifically, a human is one of the most challenging objects for a high-quality conversion, as a human body is an articulated figure and has many degrees of freedom (DOF). In addition, various styles of clothes, accessories, and hair create a very complex silhouette around the 2D human object. We propose an efficient method to estimate visually pleasing depths of a human at every frame in a monocular video. First, a 3D template model is matched to a person in a monocular video with a small number of specified user correspondences. Our pose estimation with sequential joint angular constraints reproduces a various range of human motions (i.e., spine bending) by allowing the utilization of a fully skinned 3D model with a large number of joints and DOFs. The initial depth of the 2D object in the video is assigned from the matched results, and then propagated toward areas where the depth is missing to produce a complete depth map. For the effective handling of the complex silhouettes and appearances, we introduce a partial depth propagation method based on color segmentation to ensure the detail of the results. We compared the result and depth maps painted by experienced artists. The comparison shows that our method produces viable depth maps of humans in monocular videos efficiently.

• 제어로봇시스템학회:학술대회논문집
• /
• 2003.10a
• /
• pp.669-672
• /
• 2003

3D object recognition is difficult but important in computer vision. The important thing is to understand about the relationship between a geometric structure in three dimensions and its image projection. Most 3D recognition systems construct models either manually or by training the pose and orientation of the objects. But both approaches are not satisfactory. In this paper, we focus on a commercial CAD model as a third type of model building for vision. The models are expressed in Initial Graphics Exchanges Specification(IGES) output and reconstructed in a pinhole camera coordinate.

• Proceedings of the IEEK Conference
• /
• 2002.07b
• /
• pp.1145-1148
• /
• 2002

This paper presents a new method which is calculated to use only three vanishing points in order to compute the dimensions of object and its pose from a single image of perspective projection taken by a camera and the problem of recovering 3D models from three vanishing points of box scene. Our approach is to compute only three vanishing points without this information such as the focal length, rotation matrix, and translation from images in the case of perspective projection. We assume that the object can be modeled as a linear function of a dimension vector ν. The input of reconstruction is a set of correspondences between features in the model and features in the image. To minimize each the dimensions of the parameterized models, this reconstruction of optimization can be solved by the standard nonlinear optimization techniques with a multi-start method which generates multiple starting points for the optimizer by sampling the parameter space uniformly.

• Proceedings of the KIEE Conference
• /
• 2007.04a
• /
• pp.231-233
• /
• 2007

In this paper we present a new visual approach for the robust bin-picking in a two-step concept for a vision driven automatic handling robot. The technology described here is based on two types of sensors: 3D laser scanner and CCD video camera. The geometry and pose(position and orientation) information of bin contents was reconstructed from the camera and laser sensor. these information can be employed to guide the robotic arm. A new thinning algorithm and constrained hough transform method is also explained in this paper. Consequently, the developed bin-picking demonstrate the successful operation with 3D hole object.

• Proceedings of the Korean Information Science Society Conference
• /
• 2010.06c
• /
• pp.507-511
• /
• 2010

휴먼 행동 인식은 크게 3D 모델 기반 방법과 템플릿 기반 방법으로 나눌 수 있다. 3D 모델 기반 방법은 휴먼의 포즈를 3D로 재구성한 뒤 특징을 추출하는 것으로 인식 정확도는 높으나 연산량이 많아 매우 비효율적이다. 반면 템플릿 기반의 방법은 간단하고 수행 시간이 빠르기 때문에 여러 논문들에서 채택되고 있다. 그러나 템플릿을 이용한다는 특성 때문에 시점, 행동 스타일의 변화 등에 따라 실루엣의 변화가 심해 인식 성능에 한계점을 가진다. 본 논문에서는 핵심-포즈들의 히스토그램으로 표현되는 핵심-포즈 분포와 광류의 변화를 이용하여 다중 시점에서의 휴먼 행동 인식 방법을 제안한다. 제안하는 방법은 IXMAS 데이터 셋을 이용한 실험에서 적은 수의 템플릿을 이용하면서도 평균 87.9%의 높은 인식률을 보였다.

• Journal of Korea Multimedia Society
• /
• v.24 no.11
• /
• pp.1518-1525
• /
• 2021

Due to the growth of VR industry and rise of digital twin industry, the importance of implementing 3D data same as real space is increasing. However, the fact that it requires expertise personnel and huge amount of time is a problem. In this paper, we propose a system that generates point cloud data with same shape and color as a real space, just by scanning the space. The proposed system integrates 3D geometric information from lidar and color information from stereo camera into one point cloud. Since the number of 3D points generated by lidar is not enough to express a real space with good quality, some of the pixels of 2D image generated by camera are mapped to the correct 3D coordinate to increase the number of points. Additionally, to minimize the capacity, overlapping points are filtered out so that only one point exists in the same 3D coordinates. Finally, 6DoF pose information generated from lidar point cloud is replaced with the one generated from camera image to position the points to a more accurate place. Experimental results show that the proposed system easily and quickly generates point clouds very similar to the scanned space.

• Journal of Broadcast Engineering
• /
• v.1 no.2
• /
• pp.176-187
• /
• 1996

This paper deals with the accurate estimation of 3- D pose (position and orientation) of a moving object with reference to the world frame (or robot base frame), based on a sequence of stereo images taken by cameras mounted on the end - effector of a robot manipulator. This work is an extension of the previous work[1]. Emphasis is given to the 3-D pose estimation relative to the world (or robot base) frame under the presence of not only the measurement noise in 2 - D images[ 1] but also the camera position errors due to the random noise involved in joint angles of a robot manipulator. To this end, a new set of discrete linear Kalman filter equations is derived, based on the following: 1) the orientation error of the object frame due to measurement noise in 2 - D images is modeled with reference to the camera frame by analyzing the noise propagation through 3- D reconstruction; 2) an extended Jacobian matrix is formulated by combining the result of 1) and the orientation error of the end-effector frame due to joint angle errors through robot differential kinematics; and 3) the rotational motion of an object, which is nonlinear in nature, is linearized based on quaternions. Motion parameters are computed from the estimated quaternions based on the iterated least-squares method. Simulation results show the significant reduction of estimation errors and also demonstrate an accurate convergence of the actual motion parameters to the true values.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.47 no.6
• /
• pp.39-47
• /
• 2010

A research of image-based articulated pose estimation has some problems such as detection of human feature, precise pose estimation, and real-time performance. In particular, various methods are currently presented for recovering many joints of human body. We propose the novel numerical inverse kinematics improved with the UKF(unscented Kalman filter) in order to estimate the human pose in real-time. An existing numerical inverse kinematics is required many iterations for solving the optimal estimation and has some problems such as the singularity of jacobian matrix and a local minima. To solve these problems, we combine the UKF as a tool for optimal state estimation with the numerical inverse kinematics. Combining the solution of the numerical inverse kinematics with the sampling based UKF provides the stability and rapid convergence to optimal estimate. In order to estimate the human pose, we extract the interesting human body using both background subtraction and skin color detection algorithm. We localize its 3D position with the camera geometry. Next, through we use the UKF based numerical inverse kinematics, we generate the intermediate joints that are not detect from the images. Proposed method complements the defect of numerical inverse kinematics such as a computational complexity and an accuracy of estimation.