• Proceedings of the Korea Information Processing Society Conference
• /
• 2018.10a
• /
• pp.686-689
• /
• 2018

3D 손 포즈 추정(Hand Pose Estimation, HPE)은 스마트 인간 컴퓨터 인터페이스를 위해서 중요한 기술이다. 이 연구에서는 딥러닝 방법을 기반으로 하여 단일 RGB-Depth 카메라로 촬영한 양손의 3D 손 자세를 실시간으로 인식하는 손 포즈 추정 시스템을 제시한다. 손 포즈 추정 시스템은 4단계로 구성된다. 첫째, Skin Detection 및 Depth cutting 알고리즘을 사용하여 양손을 RGB와 깊이 영상에서 감지하고 추출한다. 둘째, Convolutional Neural Network(CNN) Classifier는 오른손과 왼손을 구별하는데 사용된다. CNN Classifier 는 3개의 convolution layer와 2개의 Fully-Connected Layer로 구성되어 있으며, 추출된 깊이 영상을 입력으로 사용한다. 셋째, 학습된 CNN regressor는 추출된 왼쪽 및 오른쪽 손의 깊이 영상에서 손 관절을 추정하기 위해 다수의 Convolutional Layers, Pooling Layers, Fully Connected Layers로 구성된다. CNN classifier와 regressor는 22,000개 깊이 영상 데이터셋으로 학습된다. 마지막으로, 각 손의 3D 손 자세는 추정된 손 관절 정보로부터 재구성된다. 테스트 결과, CNN classifier는 오른쪽 손과 왼쪽 손을 96.9%의 정확도로 구별할 수 있으며, CNN regressor는 형균 8.48mm의 오차 범위로 3D 손 관절 정보를 추정할 수 있다. 본 연구에서 제안하는 손 포즈 추정 시스템은 가상 현실(virtual reality, VR), 증강 현실(Augmented Reality, AR) 및 융합 현실 (Mixed Reality, MR) 응용 프로그램을 포함한 다양한 응용 분야에서 사용할 수 있다.

• Journal of Broadcast Engineering
• /
• v.19 no.6
• /
• pp.866-876
• /
• 2014

Generating of digital hologram of video contents with computer graphics(CG) requires natural fusion of 3D information between real and virtual. In this paper, we propose the system which can fuse real-virtual 3D information naturally and fast generate the digital hologram of fused results using multiple-GPUs based computer-generated-hologram(CGH) computing part. The system calculates camera projection matrix of RGB-Depth camera, and estimates the 3D information of virtual object. The 3D information of virtual object from projection matrix and real space are transmitted to Z buffer, which can fuse the 3D information, naturally. The fused result in Z buffer is transmitted to multiple-GPUs based CGH computing part. In this part, the digital hologram of fused result can be calculated fast. In experiment, the 3D information of virtual object from proposed system has the mean relative error(MRE) about 0.5138% in relation to real 3D information. In other words, it has the about 99% high-accuracy. In addition, we verify that proposed system can fast generate the digital hologram of fused result by using multiple GPUs based CGH calculation.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2019.05a
• /
• pp.445-446
• /
• 2019

Recently, attention has been focused on mixed reality (MR) technology, which provides an experience that can not be realized in reality by fusing virtual information into the real world. Mixed reality has the advantage of having excellent interaction with reality and maximizing immersion feeling. In this paper, we propose a method to acquire a point cloud for the production of mixed reality contents using multiple Depth and RGB camera system.

• Smart Media Journal
• /
• v.1 no.3
• /
• pp.15-21
• /
• 2012

Depth information is necessary for various three dimensional contents generation. Depth acquisition techniques can be categorized broadly into two approaches: active, passive depth sensors depending on how to obtain depth information. In this paper, we take a look at several ways of depth acquirement. We present not only depth acquisition methods using discussed ways, but also hybrid methods which combine both approaches to compensate for drawbacks of each approach. Furthermore, we introduce several matching cost functions and post-processing techniques to enhance the temporal consistency and reduce flickering artifacts and discomforts of users caused by inaccurate depth estimation in 3D video.

• Journal of Broadcast Engineering
• /
• v.25 no.5
• /
• pp.798-807
• /
• 2020

As technologies using digital camera have been developed, 3D images can be constructed from the pictures captured by using multiple cameras. The 3D image data is represented in a form of point cloud which consists of 3D coordinate of the data and the related attributes. Various techniques have been proposed to construct the point cloud data. Among them, Structure-from-Motion (SfM) and Multi-view Stereo (MVS) are examples of the image-based technologies in this field. Based on the conventional research, the point cloud data generated from SfM and MVS may be sparse because the depth information may be incorrect and some data have been removed. In this paper, we propose an efficient algorithm to enhance the point cloud so that the density of the generated point cloud increases. Simulation results show that the proposed algorithm outperforms the conventional algorithms objectively and subjectively.

• Journal of Digital Convergence
• /
• v.14 no.1
• /
• pp.189-194
• /
• 2016

Recently, there is increasing demand for image processing technology while activated the use of equipments such as camera, camcorder and CCTV. In particular, research and development related to 3D image technology using the depth camera such as Kinect sensor has been more activated. Kinect sensor is a high-performance camera that can acquire a 3D human skeleton structure via a RGB, skeleton and depth image in real-time frame-by-frame. In this paper, we develop a system. This system captures the motion of a 3D human skeleton structure using the Kinect sensor. And this system can be stored by selecting the motion file format as trc and bvh that is used for general purposes. The system also has a function that converts TRC motion captured format file into BVH format. Finally, this paper confirms visually through the motion capture data viewer that motion data captured using the Kinect sensor is captured correctly.

• Journal of Broadcast Engineering
• /
• v.25 no.3
• /
• pp.439-448
• /
• 2020

In this paper, we propose a modified optimization algorithm for point cloud matching of multi-view RGB-D cameras. In general, in the computer vision field, it is very important to accurately estimate the position of the camera. The 3D model generation methods proposed in the previous research require a large number of cameras or expensive 3D cameras. Also, the methods of obtaining the external parameters of the camera through the 2D image have a large error. In this paper, we propose a matching technique for generating a 3D point cloud and mesh model that can provide omnidirectional free viewpoint using 8 low-cost RGB-D cameras. We propose a method that uses a depth map-based function optimization method with RGB images and obtains coordinate transformation parameters that can generate a high-quality 3D model without obtaining initial parameters.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2019.06a
• /
• pp.234-235
• /
• 2019

본 논문은 실사 객체를 360도 전방위에서 관찰이 가능한 3D 그래픽 모델로 변환하는 시스템에서 뼈대를 추출하는 방법을 제시한다. 각 카메라로부터 촬영된 텍스쳐 영상을 이용하여 뼈대를 추출하고, 깊이 정보로부터 얻어진 포인트 클라우드 정보를 이용하여 뼈대 정보를 정합, 보정하는 과정을 수행한다. 카메라로부터 촬영된 텍스쳐 영상에 대해 딥러닝 기술 등을 이용하여 뼈대를 획득한다. 텍스쳐 영상으로부터 획득된 뼈대 정보는 동일 위치에서 획득된 외부 파라미터를 이용하여 월드좌표계로 변환하여 공간상에 위치시킨다. 이러한 과정을 모든 카메라로부터 획득된 뼈대 정보에 동일하게 적용함으로써 모든 뼈대 정보를 공간상에 표현하여 최종적인 뼈대 정보를 추출하는 방법을 제시한다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.270-272
• /
• 2022

한의 복진은 복부를 검사하기 위해 수행되지만 정량화되지는 않았다. 이 연구의 목표는 소화불량의 주요 증상인 흉협고만이 있는 그룹과 아닌 그룹 사이에 유의하게 차이나는 복부 측정기기의 변수를 식별하는 것이다. 정량적인 진단을 위해 규칙에 따라 구획한 복부를 적외선 열화상 카메라, 디지털 압통기, 3D 카메라 및 디지털 청진기를 포함한 기기로 측정하였다. 연구방법으로 임상연구를 수행하여 한의사들이 진단한 복부 증상인 흉협고만과의 일치도를 조사하였다. 기기 측정 중 깊이, 압력, 깊이에 대한 압력의 비율은 흉협고만 그룹이 비 흉협고만 그룹보다 유의하게 작았다. 따라서 물리적 압통 특성이 감소하고, 복부 경직도가 감소하며, 민감도가 증가했다. 좌측과 우측 늑골 사이의 거리, 흉늑골 각도는 흉협고만 환자에서 유의하게 더 컸다. 또한, 깊이 차이, 표면 법선 벡터 및 깊이 값 사이의 각도 차이는 흉협고만 그룹에서 대부분 작았다. 복부 측정기기는 다양한 질환 및 증상에 사용될 것으로 기대한다.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.38C no.5
• /
• pp.409-419
• /
• 2013

Kinect released by Microsoft in November 2010 is a motion sensing camera in xbox360 and gives depth and color images. However, Kinect camera also generates holes and noise around object boundaries in the obtained images because it uses infrared pattern. Also, boundary flickering phenomenon occurs. Therefore, we propose a real-time virtual-view video synthesis algorithm which results in a high-quality virtual view by solving these problems. In the proposed algorithm, holes around the boundary are filled by using the joint bilateral filter. Color image is converted into intensity image and then flickering pixels are searched by analyzing the variation of intensity and depth images. Finally, boundary flickering phenomenon can be reduced by converting values of flickering pixels into the maximum pixel value of a previous depth image and virtual views are generated by applying 3D warping technique. Holes existing on regions that are not part of occlusion region are also filled with a center pixel value of the highest reliability block after the final block reliability is calculated by using a block based gradient searching algorithm with block reliability. The experimental results show that the proposed algorithm generated the virtual view image in real-time.