• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2015.11a
• /
• pp.169-172
• /
• 2015

영상회의 장치에서 눈맞춤 영상을 생성하기 위해 어파인 변환(affine transformation)을 이용하면 반올림 오차(round-off error) 때문에 홀이 발생한다. 이러한 홀을 채우려면 홀 영역을 가리키는 홀 채움 마스크가 필요하다. 홀 채움은 보통 홀 채움 마스크를 참조하여 홀이 아닌 이웃 화소값들을 기반으로 손상된 영상을 복원하는 작업이다. 따라서 홀 영역을 정확히 검출하고 적당한 개수의 이웃 화소값을 참조해야 자연스러운 홀 채움 영상을 생성할 수 있다. 한편, 눈맞춤 영상을 생성할 때 얼굴 특징점을 이용해 얼굴 변환 마스크를 만들고, 얼굴 변환 마스크에만 어파인 변환을 수행한다. 이 논문에서는 얼굴 특징점에도 어파인 변환을 수행하고 수정된 얼굴 변환 마스크를 획득하여 정확한 홀 채움 마스크를 구한다. 또한, 홀 채움 마스크에서 레이블링을 수행하여 큰 홀 영역을 제거한다. 마지막으로, 어파인 변환을 수행할 때 기존 영상의 좌표값을 이용하여 자연스러운 홀 채움 영상을 생성한다. 제안하는 방법으로 홀 채움을 수행한 결과, 연속적인 눈맞춤 동영상에서 이웃값들을 참조하여 홀 채움을 수행한 영상보다 자연스러움을 확인했다.

• Smart Media Journal
• /
• v.3 no.4
• /
• pp.28-34
• /
• 2014

Recently the depth-image-based rendering (DIBR) method is usually used in 3D image application filed. Virtual view image is created by using a known view with associated depth map to make a virtual view point which did not taken by the camera. But, disocclusion area occur because the virtual view point is created using a depth image based image 3D warping. To remove those kind of disocclusion region, many hole filling methods are proposed until now. Constant color region searching, horizontal interpolation, horizontal extrapolation, and variational inpainting techniques are proposed as a hole filling methods. But when using those hole filling method some problem occurred. The different types of annoying artifacts are appear in texture region hole filling procedure. In this paper to solve those problem, the multi-directional extrapolation method is newly proposed for efficiency of expanded hole filling performance. The proposed method is efficient when performing hole filling which complex texture background region. Consideration of directionality for hole filling method use the hole neighbor texture pixel value when estimate the hole pixel value. We can check the proposed hole filling method can more efficiently fill the hole region which generated by virtual view synthesis result.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2014.11a
• /
• pp.93-96
• /
• 2014

본 논문에서는 가상시점 영상을 생성할 때 발생하는 홀 영역을 효율적으로 채우는 방법을 제안한다. 가상시점 영상을 생성하려면 우선 깊이 영상에 대해 3차원 워핑을 수행한 뒤, 이때 발생하는 작은 홀을 미디언 필터를 이용하여 적절히 채워야 한다. 홀이 채워진 깊이 영상을 기반으로 하여 입력받은 참조 컬러 영상에 대해 3차원 워핑을 수행하여 가상 위치에 새로운 시점 영상을 생성하게 된다. 이때 또한 마찬가지로 3차원 워핑을 수행하기 때문에 홀 영역이 발생하게 된다. 텍스쳐 영상을 워핑하여 새로운 좌표계로 옮긴 영상은 주변 컬러 화소들과의 관계들을 가지고 있다. 텍스쳐 영상을 워핑한 결과 영상에서 발생하는 홀 영역을 채우기 위해 방향성을 고려한 홀 채움 방법을 사용한다. 홀 주변 화소 영역의 값들을 홀을 채우게 될 후보 화소 값으로 설정한 뒤, 각각의 화소값에 대해 비용값을 계산한다. 이때 가장 적은 비용값을 갖게 하는 주변 화소 값을 해당 영역의 홀 채움 값으로 사용하게 된다. 좌영상과 우영상을 워핑할 때 발생하는 홀 영역의 위치가 각각 다르게 나타난다. 홀 영역은 배경화소 값을 이용해 채울 경우 자연스러운 결과를 보인다. 배경화소 값을 이용하기 위해 좌영상과 우영상에 따른 새로운 홀 스캔 방향 또한 제안한다. 능동적으로 홀 스캔 방향을 선택하여 홀 주변 화소값들을 스캔해가며 워핑 결과 발생하는 홀 영역을 효율적으로 채우게 된다. 결과적으로 제안한 방법을 통하여 생성된 가상시점 영상의 화질이 좋아지는 결과를 확인할 수 있었다.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2017.11a
• /
• pp.133-135
• /
• 2017

본 논문에서는 스테레오 영상을 이용하여 외삽 시점 영상 생성 시 발생하는 홀을 채우는 방법을 제안한다. 스테레오 영상에 3D 워핑을 이용하여 다수의 시점을 생성할 수 있다. 하지만 이 방법은 보이지 않는 시점에서의 영역을 완벽히 복원할 수 없기 때문에 필연적으로 홀이 발생한다. 홀을 채우기 위해 먼저 홀 영역의 경계를 Random Walks 알고리즘을 이용하여 전경과 배경으로 구분한다. 그리고 홀을 배경 성분에 해당하는 영역만을 이용하여 채우게 된다. 홀 채움 과정에서는 패치 내의 홀의 비율과 컬러와 깊이 영상의 텍스처에 대한 복잡도를 정의하고 패치 별로 우선순위를 계산하여 높은 순위의 패치로 홀을 채우게 된다. 실험 결과 제안하는 기법이 홀을 효과적으로 채우는 것을 확인하였다.

• Journal of IKEEE
• /
• v.20 no.1
• /
• pp.31-44
• /
• 2016

This paper presents a hole-filling algorithm based on extrapolating spatial-temporal background information used in view synthesis for free-viewpoint television. A new background codebook is constructed and updated in order to extract reliable temporal background information. In addition, an estimation of spatial local background values is conducted to discriminate an adaptive boundary between the background region and the foreground region as well as to update the information about the hole region. The holes then are filled by combining the spatial background information and the temporal background information. In addition, an exemplar-based inpainting technique is used to fill the rest of holes, in which a priority function using background-depth information is defined to determine the order in which the holes are filled. The experimental results demonstrated that the proposed algorithm outperformed the other comparative methods about average 0.3-0.6 dB, and that it synthesized satisfactory views regardless of video characteristics and type of hole region.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.54 no.8
• /
• pp.67-80
• /
• 2017

This paper presents a hole-filling method using extrapolated spatio-temporal background information to obtain a synthesized view. A new temporal background model using non-overlapped patch based background codebook is introduced to extrapolate temporal background information In addition, a depth-map driven spatial local background estimation is addressed to define spatial background constraints that represent the lower and upper bounds of a background candidate. Background holes are filled by comparing the similarities between the temporal background information and the spatial background constraints. Additionally, a depth map-based ghost removal filter is described to solve the problem of the non-fit between a color image and the corresponding depth map of a virtual view after 3-D warping. Finally, an inpainting is applied to fill in the remaining holes with the priority function that includes a new depth term. The experimental results demonstrated that the proposed method led to results that promised subjective and objective improvement over the state-of-the-art methods.

• Journal of the Korea Computer Graphics Society
• /
• v.26 no.3
• /
• pp.39-48
• /
• 2020

In this paper, we propose a new technique for filling holes in triangular mesh. First, arbitrary shaped holes are detected. Second, source and target hole patches are generated through triangulation, refinement, fairing, and smoothing. Finally, the shape difference between the two patches is analyzed and a patch with enhanced features is obtained through blending between patches. The effectiveness of the proposed technique is demonstrated by applying the hole filling technique to the triangular mesh model with various shaped holes.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.17 no.12
• /
• pp.2927-2936
• /
• 2013

In this paper, we propose a noise reduction and hole removal algorithm in order to improve the quality of depth images when they are used for creating 3D contents. In the proposed algorithm, the depth image and the corresponding color image are both used. First, an intensity image is generated by converting the RGB color space into the HSI color space. By estimating the difference of distance and depth between reference and neighbor pixels from the depth image and difference of intensity values from the color image, they are used to remove noise in the proposed algorithm. Then, the proposed hole filling method fills the detected holes with the difference of euclidean distance and intensity values between reference and neighbor pixels from the color image. Finally, we apply a parallel structure of GPGPU to the proposed algorithm to speed-up its processing time for real-time applications. The experimental results show that the proposed algorithm performs better than other conventional algorithms. Especially, the proposed algorithm is more effective in reducing edge blurring effect and removing noise and holes.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.37 no.8A
• /
• pp.679-688
• /
• 2012

In this paper, performance improved hole-filling algorithm including boundary noise removing pre-process which can be used for an arbitrary view synthesis with given two views is proposed. Boundary noise usually occurs because of the boundary mismatch between the reference image and depth map and common-hole is defined as the occluded region. These boundary noise and common-hole created while synthesizing a virtual view result in some defects and they are usually very difficult to be completely recovered by using only given two images as references. The spiral weighted average algorithm gives a clear boundary of each object by using depth information and the gradient searching algorithm is able to preserve details. In this paper, we combine these two algorithms by using a weighting factor ${\alpha}$ to reflect the strong point of each algorithm effectively in the virtual view synthesis process. The experimental results show that the proposed algorithm performs much better than conventional algorithms.

• The Journal of the Korea Contents Association
• /
• v.15 no.2
• /
• pp.21-30
• /
• 2015

Image coding based on mixed raster content model generates don't care regions (DCR) in foreground and background layers, and its overall coding performance is greatly affected by region filling methods for DCRs. Most conventional methods for DCR filling fail in utilizing the local signal properties in hole regions and thus the high frequency components in non-DCR regions are reflected into DCR after signal filling. In addition, further high frequency components are induced to the filled signal because of signal discontinuities in the boundary of DCR. To solve this problem, a new DCR filling algorithm using the priority-based adaptive region growing is proposed in this paper. The proposed method uses the watershed algorithm and the flooding priority of each pixel for region filling is determined from the degree of smoothness in the neighborhood area. By growing the filled region into DCR based on the computed priority, the expansion of high-textured area can be minimized which can improve the overall coding performance. Experimental results show that the proposed method outperforms conventional algorithms.