• Journal of the Institute of Electronics and Information Engineers
• /
• v.54 no.8
• /
• pp.67-80
• /
• 2017

This paper presents a hole-filling method using extrapolated spatio-temporal background information to obtain a synthesized view. A new temporal background model using non-overlapped patch based background codebook is introduced to extrapolate temporal background information In addition, a depth-map driven spatial local background estimation is addressed to define spatial background constraints that represent the lower and upper bounds of a background candidate. Background holes are filled by comparing the similarities between the temporal background information and the spatial background constraints. Additionally, a depth map-based ghost removal filter is described to solve the problem of the non-fit between a color image and the corresponding depth map of a virtual view after 3-D warping. Finally, an inpainting is applied to fill in the remaining holes with the priority function that includes a new depth term. The experimental results demonstrated that the proposed method led to results that promised subjective and objective improvement over the state-of-the-art methods.

• Smart Media Journal
• /
• v.1 no.3
• /
• pp.29-35
• /
• 2012

A depth image-based rendering (DIBR) technique is one of the rendering processes of virtual views with a color image and the corresponding depth map. The most important issue of DIBR is that the virtual view has no information at newly exposed areas, so called dis-occlusion. In this paper, we propose an intermediate view generation algorithm using the Kinect depth camera that utilizes the infrared structured light. After we capture a color image and its corresponding depth map, we pre-process the depth map. The pre-processed depth map is warped to the virtual viewpoint and filtered by median filtering to reduce the truncation error. Then, the color image is back-projected to the virtual viewpoint using the warped depth map. In order to fill out the remaining holes caused by dis-occlusion, we perform a background-based image in-painting operation. Finally, we obtain the synthesized image without any dis-occlusion. From experimental results, we have shown that the proposed algorithm generated very natural images in real-time.

• Journal of IKEEE
• /
• v.20 no.1
• /
• pp.31-44
• /
• 2016

This paper presents a hole-filling algorithm based on extrapolating spatial-temporal background information used in view synthesis for free-viewpoint television. A new background codebook is constructed and updated in order to extract reliable temporal background information. In addition, an estimation of spatial local background values is conducted to discriminate an adaptive boundary between the background region and the foreground region as well as to update the information about the hole region. The holes then are filled by combining the spatial background information and the temporal background information. In addition, an exemplar-based inpainting technique is used to fill the rest of holes, in which a priority function using background-depth information is defined to determine the order in which the holes are filled. The experimental results demonstrated that the proposed algorithm outperformed the other comparative methods about average 0.3-0.6 dB, and that it synthesized satisfactory views regardless of video characteristics and type of hole region.

• Journal of the Institute of Convergence Signal Processing
• /
• v.8 no.4
• /
• pp.249-254
• /
• 2007

This paper describes a technique for detecting the boundary of the optic disk in digital image of the retina using inward and outward curve evolution. Optic disk boundary offers medical information about glaucoma progresses. For accurate boundary detection, image inpainting based on PDE removes blood vessels crossing the optic disk. For removing noises and preserving boundary of optic disk in image inpainting process, the anisotropic diffusion filtering is developed. After pre-processing, the optic disk boundary is determined using inward and outward curve evolution. Experimental results show that blurring effect of original region and optic disk boundary is reduced considerably. By the proposed method, we can detect correct disk boundary compare to conventional method.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.44 no.5
• /
• pp.54-63
• /
• 2007

The purpose of this study is to present a new energy minimization method for image completion with hierarchical approach. The goal of image completion is to fill in missing part in a possibly large region of an image so that a visually plausible outcome is obtained. An exemplar-based Markov Random Field Modeling(MRF) is proposed in this paper. This model can deal with following problems; detection of global features, flexibility on environmental changes, reduction of computational cost, and generic extension to other related domains such as image inpainting. We use the Priority Belief Propagation(Priority-BP) which is a kind of Belief propagation(BP) algorithms for the optimization of MRF. We propose the hierarchical Priority-BP that reduces the number of nodes in MRF and to apply hierarchical propagation of messages for image completion. We show that our approach which uses hierarchical Priority-BP algorithm in image completion works well on a number of examples.

• Smart Media Journal
• /
• v.3 no.4
• /
• pp.28-34
• /
• 2014

Recently the depth-image-based rendering (DIBR) method is usually used in 3D image application filed. Virtual view image is created by using a known view with associated depth map to make a virtual view point which did not taken by the camera. But, disocclusion area occur because the virtual view point is created using a depth image based image 3D warping. To remove those kind of disocclusion region, many hole filling methods are proposed until now. Constant color region searching, horizontal interpolation, horizontal extrapolation, and variational inpainting techniques are proposed as a hole filling methods. But when using those hole filling method some problem occurred. The different types of annoying artifacts are appear in texture region hole filling procedure. In this paper to solve those problem, the multi-directional extrapolation method is newly proposed for efficiency of expanded hole filling performance. The proposed method is efficient when performing hole filling which complex texture background region. Consideration of directionality for hole filling method use the hole neighbor texture pixel value when estimate the hole pixel value. We can check the proposed hole filling method can more efficiently fill the hole region which generated by virtual view synthesis result.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.48 no.6
• /
• pp.48-60
• /
• 2011

Nowadays, the 3D community is actively researching on 3D imaging and free-viewpoint video (FVV). The free-viewpoint rendering in multi-view video, virtually move through the scenes in order to create different viewpoints, has become a popular topic in 3D research that can lead to various applications. However, there are restrictions of cost-effectiveness and occupying large bandwidth in video transmission. An alternative to solve this problem is to generate virtual views using a single texture image and a corresponding depth image. A critical issue on generating virtual views is that the regions occluded by the foreground (FG) objects in the original views may become visible in the synthesized views. Filling this disocclusions (holes) in a visually plausible manner determines the quality of synthesis results. In this paper, a new approach for handling disocclusions using depth based inpainting algorithm in synthesized views is presented. Patch based non-parametric texture synthesis which shows excellent performance has two critical elements: determining where to fill first and determining what patch to be copied. In this work, a noise-robust filling priority using the structure tensor of Hessian matrix is proposed. Moreover, a patch matching algorithm excluding foreground region using depth map and considering epipolar line is proposed. Superiority of the proposed method over the existing methods is proved by comparing the experimental results.