• 한국HCI학회:학술대회논문집
• /
• 2007.02a
• /
• pp.96-102
• /
• 2007

본 논문에서는 자연스러운 증강현실 환경을 위하여 패치 기반의 텍스쳐 합성을 통한 마커 은닉 방법을 제안한다. 증강현실에서 카메라의 자세를 구하기 위한 보편적인 방법은 음영 대비가 뚜렷한 정사각형의 마커를 사용하는 것이다. 이러한 인위적인 마커의 사용은 물체의 인식과 추적을 용이하게 하지만 증강된 장면의 실감성을 감소시켜 사용성 저하를 유발하기도 한다. 제안된 마커 은닉 방법은 실시간성을 보장하면서, 배경 텍스쳐의 전역적인 특성을 유지하고, 주변 환경의 변화에 유연하다.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2016.11a
• /
• pp.47-50
• /
• 2016

객체를 추적하는 기술은 컴퓨터 비전 분야에서 활발히 연구되고 있는 분야 중 하나이다. 그 중 고정된 단일 카메라를 이용한 객체 추적 기술은 비디오 감시(Surveillance) 등에서 활용되고 있다. 고정된 카메라 환경에서 객체를 추적하는 방법 중 배경 모델링(Background Modeling)을 이용한 방법은 간단하면서도 널리 사용되는 방법 중 하나이다. 객체의 움직임이나 특징을 분석하여 배경 모델을 생성한 후 배경 정보를 이용하여 전경을 분리하면 쉽게 객체를 추출할 수 있다. 그러나 객체의 움직임이 적은 경우 해당 영역에서의 배경 모델은 정확하게 생성될 수 없다. 배경 모델을 학습하는 동안 객체가 충분이 움직이면 이런 문제를 해결할 수 있으나 객체가 움직이기 전까지는 오류가 지속된다. 이런 문제를 해결하기 위해 본 논문에서는 인페인팅(Inpainting)을 이용하여 움직임이 적은 영역을 보정하여 정확한 배경 모델을 생성하는 방법을 제안한다. 배경 모델을 생성한 후 객체로 식별할 수 있는 후보 영역을 식별한다. 선정된 영역들 중 사용자가 객체로 판단되는 영역을 선택하여 해당 영역에 대해 인페인팅으로 화소값 및 가중치들을 보정한다. 보정된 영상으로 배경 모델링을 수행하면 움직임이 적은 영역에 대해서도 효과적으로 배경 모델을 생성 할 수 있다.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2012.11a
• /
• pp.25-28
• /
• 2012

본 논문에서는 키넥트(Kinect)에서 획득한 깊이 영상의 품질을 향상시키는 기법을 제안한다. 키넥트는 마이크로소프트사에서 출시한 카메라로 깊이 영상과 컬러 영상을 획득 할 수 있다. 하지만 적외선 패턴을 이용한 깊이 영상의 획득 방법의 한계로 인해 객체의 경계 주변으로 홀 및 잡음이 생긴다. 따라서 정확한 깊이 영상을 얻기 위해서는 깊이 영상의 품질 향상이 필수적이다. 일반적으로 깊이 영상의 홀을 채울 때, 인페인팅(inpainting) 또는 결합형 양방향 필터(joint bilateral filter) 등의 기법을 사용한다. 그러나 이러한 기법들의 경우 한 장의 영상만을 이용하기 때문에 객체 경계 주변의 흔들림 현상을 보정할 수 없다. 제안하는 기법에서는 먼저 수행속도가 빠른 가우시안 필터를 이용하여 경계 주변의 홀을 채운다. 이전 프레임의 컬러 영상을 그레이 영상으로 변환한 다음에 그레이 영상과 깊이 영상의 값의 변화를 분석하여 흔들림 화소를 찾아 이전 깊이 영상들 중 최대 화소 값으로 변환함으로써 깊이 영상의 경계 흔들림 현상을 줄일 수 있다. 실험을 통해 제안하는 기법이 기존의 방법들 보다 우수하다는 것을 확인하였다.

• Smart Media Journal
• /
• v.1 no.3
• /
• pp.29-35
• /
• 2012

A depth image-based rendering (DIBR) technique is one of the rendering processes of virtual views with a color image and the corresponding depth map. The most important issue of DIBR is that the virtual view has no information at newly exposed areas, so called dis-occlusion. In this paper, we propose an intermediate view generation algorithm using the Kinect depth camera that utilizes the infrared structured light. After we capture a color image and its corresponding depth map, we pre-process the depth map. The pre-processed depth map is warped to the virtual viewpoint and filtered by median filtering to reduce the truncation error. Then, the color image is back-projected to the virtual viewpoint using the warped depth map. In order to fill out the remaining holes caused by dis-occlusion, we perform a background-based image in-painting operation. Finally, we obtain the synthesized image without any dis-occlusion. From experimental results, we have shown that the proposed algorithm generated very natural images in real-time.

• The KIPS Transactions:PartB
• /
• v.17B no.1
• /
• pp.47-54
• /
• 2010

Old archived film shows two major defects: line scratch and blobs. In this paper, we present a design and implementation of an automatic video restoration system for line scratches observed in archived film. We use autoregressive (AR) image model because we can make stochastic and specifically autoregressive image generation process with our PAST-PRESENT model and Sampling Pattern. We designed locality maximizing scanning pattern, which can generate nearly stationary time-like series of pixels, which is a strong requirement for a stochastic series to be autoregressive. The sampled pixel series undergoes filtering and model fitting using Durbin-Levinson algorithm before interpolation process. We designed three-stage film restoration system, which includes (1) film acquisition from VHS tapes, (2) simple line scratch detection and restoration, and (3) manual blob identification and sophisticated inpainting scheme. We implemented film acquisition and simple inpainting scheme on Texas Instruments DSP board TMS320DM642 EVM, and implemented our AR inpainting scheme on PC for sophisticated restoration. We experimented our scheme with two old Korean films: "Viva Freedom" and "Robot Tae-Kwon-V", and the experimental results show that our scheme improves Bertalmio's scheme for subjective quality (MOS), objective quality (PSNR), and especially restoration ratio (RR), which reflects how much similar to the manual inpainting results.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2009.11a
• /
• pp.349-350
• /
• 2009

In this paper, we describe an error concealment techniques based on image inpainting for the image impairments due to the packet loss. Image inpainting is to remove or restore the damaged sections from the images, which is usually old images, paintings, or video films. Inpainting has a long history which goes back to the era when the paintings come out. Manual inpainting is no more used, and we can use digital inpainting for the digitally impaired images and video sequences. In this paper, we review the error concealment techniques for the packet loss recovery and propose our inpainting based image impairment recovery scheme for video communication over packet networks.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.54 no.8
• /
• pp.67-80
• /
• 2017

This paper presents a hole-filling method using extrapolated spatio-temporal background information to obtain a synthesized view. A new temporal background model using non-overlapped patch based background codebook is introduced to extrapolate temporal background information In addition, a depth-map driven spatial local background estimation is addressed to define spatial background constraints that represent the lower and upper bounds of a background candidate. Background holes are filled by comparing the similarities between the temporal background information and the spatial background constraints. Additionally, a depth map-based ghost removal filter is described to solve the problem of the non-fit between a color image and the corresponding depth map of a virtual view after 3-D warping. Finally, an inpainting is applied to fill in the remaining holes with the priority function that includes a new depth term. The experimental results demonstrated that the proposed method led to results that promised subjective and objective improvement over the state-of-the-art methods.

• Journal of IKEEE
• /
• v.20 no.1
• /
• pp.31-44
• /
• 2016

This paper presents a hole-filling algorithm based on extrapolating spatial-temporal background information used in view synthesis for free-viewpoint television. A new background codebook is constructed and updated in order to extract reliable temporal background information. In addition, an estimation of spatial local background values is conducted to discriminate an adaptive boundary between the background region and the foreground region as well as to update the information about the hole region. The holes then are filled by combining the spatial background information and the temporal background information. In addition, an exemplar-based inpainting technique is used to fill the rest of holes, in which a priority function using background-depth information is defined to determine the order in which the holes are filled. The experimental results demonstrated that the proposed algorithm outperformed the other comparative methods about average 0.3-0.6 dB, and that it synthesized satisfactory views regardless of video characteristics and type of hole region.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.44 no.5
• /
• pp.54-63
• /
• 2007

The purpose of this study is to present a new energy minimization method for image completion with hierarchical approach. The goal of image completion is to fill in missing part in a possibly large region of an image so that a visually plausible outcome is obtained. An exemplar-based Markov Random Field Modeling(MRF) is proposed in this paper. This model can deal with following problems; detection of global features, flexibility on environmental changes, reduction of computational cost, and generic extension to other related domains such as image inpainting. We use the Priority Belief Propagation(Priority-BP) which is a kind of Belief propagation(BP) algorithms for the optimization of MRF. We propose the hierarchical Priority-BP that reduces the number of nodes in MRF and to apply hierarchical propagation of messages for image completion. We show that our approach which uses hierarchical Priority-BP algorithm in image completion works well on a number of examples.

• Smart Media Journal
• /
• v.3 no.4
• /
• pp.28-34
• /
• 2014

Recently the depth-image-based rendering (DIBR) method is usually used in 3D image application filed. Virtual view image is created by using a known view with associated depth map to make a virtual view point which did not taken by the camera. But, disocclusion area occur because the virtual view point is created using a depth image based image 3D warping. To remove those kind of disocclusion region, many hole filling methods are proposed until now. Constant color region searching, horizontal interpolation, horizontal extrapolation, and variational inpainting techniques are proposed as a hole filling methods. But when using those hole filling method some problem occurred. The different types of annoying artifacts are appear in texture region hole filling procedure. In this paper to solve those problem, the multi-directional extrapolation method is newly proposed for efficiency of expanded hole filling performance. The proposed method is efficient when performing hole filling which complex texture background region. Consideration of directionality for hole filling method use the hole neighbor texture pixel value when estimate the hole pixel value. We can check the proposed hole filling method can more efficiently fill the hole region which generated by virtual view synthesis result.