• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.37 no.8A
• /
• pp.679-688
• /
• 2012

In this paper, performance improved hole-filling algorithm including boundary noise removing pre-process which can be used for an arbitrary view synthesis with given two views is proposed. Boundary noise usually occurs because of the boundary mismatch between the reference image and depth map and common-hole is defined as the occluded region. These boundary noise and common-hole created while synthesizing a virtual view result in some defects and they are usually very difficult to be completely recovered by using only given two images as references. The spiral weighted average algorithm gives a clear boundary of each object by using depth information and the gradient searching algorithm is able to preserve details. In this paper, we combine these two algorithms by using a weighting factor ${\alpha}$ to reflect the strong point of each algorithm effectively in the virtual view synthesis process. The experimental results show that the proposed algorithm performs much better than conventional algorithms.

• Electronics and Telecommunications Trends
• /
• v.35 no.5
• /
• pp.112-122
• /
• 2020

In 3D computer graphics, a depth map is an image that provides information related to the distance from the viewpoint to the subject's surface. Stereo sensors, depth cameras, and imaging systems using an active illumination system and a time-resolved detector can perform accurate depth measurements with their own light sources. The 3D image information obtained through the depth map is useful in 3D modeling, autonomous vehicle navigation, object recognition and remote gesture detection, resolution-enhanced medical images, aviation and defense technology, and robotics. In addition, the depth map information is important data used for extracting and restoring multi-view images, and extracting phase information required for digital hologram synthesis. This study is oriented toward a recent research trend in deep learning-based 3D data analysis methods and depth map information extraction technology using a convolutional neural network. Further, the study focuses on 3D image processing technology related to digital hologram and multi-view image extraction/reconstruction, which are becoming more popular as the computing power of hardware rapidly increases.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2008.11a
• /
• pp.113-116
• /
• 2008

본 논문에서는 다시점 비디오(multi-view video)에서 보다 다양한 시점을 제공하기 위한 가상 시점 비디오 생성 기법을 제안한다. 제안하는 가상 시점 비디오 생성 기법은 우선적으로 대칭적 신뢰 전파 기법(symmetric belief propagation)을 기반으로, 각 시점의 깊이 정보 및 폐색 영역(occlusion region)을 추출하기 위해서 에너지를 최소화한다. 추출된 깊이 정보 및 에너지를 이용하여 참조하는 시점 간의 가중치를 적용하여, 새로운 가상 시점의 비디오를 생성하고, 추출된 폐색 영역의 값을 이용하여, 가상 시점의 비디오를 보정하는 가상 시점 비디오 생성 기법을 제안한다. 또한 제안하는 알고리즘을 한정된 중간 시점 영상에서 임의의 가상 시점으로 확장하여, 임의의 두 시점 간의 자유로운 시점(free-view point)을 제공함을 확인한다. 실험을 통하여 제안하는 기법이 다시점 비디오에서 높은 화질의 가상 시점 비디오를 제공함을 확인한다.

• Journal of Broadcast Engineering
• /
• v.18 no.6
• /
• pp.859-871
• /
• 2013

In this paper, we propose an algorithm that generates multi-view images by grid-mesh based image domain warping using occlusion mask and various image features obtained from the stereoscopic images. In the proposed algorithm, we first extract image saliency map, line segments and disparity saliency map from stereo images and then get them through a process that improves the quality of extracted features. This process is accomplished in two steps. In the first step, reliability of disparity saliency map on object boundary regions is enhanced by using occlusion information. And in the second step, we enhance the quality of image features in terms of temporal consistency by using temporal consistency information for stereo images. With these enhanced features, multi-view images are generated by grid-mesh based image domain warping technique. Experimental results show that the proposed algorithm performs better than existing algorithms in terms of visual quality.

• ETRI Journal
• /
• v.38 no.6
• /
• pp.1114-1123
• /
• 2016

Multi-view video plus depth (MVD) has been widely used owing to its effectiveness in three-dimensional data representation. Using MVD, color videos with only a limited number of real viewpoints are compressed and transmitted along with captured or estimated depth videos. Because the synthesized views are generated from decoded real views, their original reference views do not exist at either the transmitter or receiver. Therefore, it is challenging to define an efficient metric to evaluate the quality of synthesized images. We propose a novel metric-the reduced-reference quality metric. First, the effects of depth distortion on the quality of synthesized images are analyzed. We then employ the high correlation between the local depth distortions and local color characteristics of the decoded depth and color images, respectively, to achieve an efficient depth quality metric for each real view. Finally, the objective quality metric of the synthesized views is obtained by combining all the depth quality metrics obtained from the decoded real views. The experimental results show that the proposed quality metric correlates very well with full reference image and video quality metrics.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.36 no.12C
• /
• pp.738-743
• /
• 2011

For the high quality 3-D broadcasting, depth maps are important data. Although commercially available depth cameras capture high-accuracy depth maps in real time, their resolutions are much smaller than those of the corresponding color images due to technical limitations. In this paper, we propose the depth map up-sampling method using a high-resolution color image and a low-resolution depth map. We define a random walk probability model in an operation unit which has nearest seed pixels. The proposed method is appropriate to match boundaries between the color image and the depth map. Experimental results show that our method enhances the depth map resolution successfully.

• Journal of Information Display
• /
• v.6 no.4
• /
• pp.26-34
• /
• 2005

In this paper, a new rectification scheme to transform the uncalibrated stereo image pair into the calibrated one is suggested and its performance is analyzed by applying this scheme to the reconstruction of the intermediate views for multi-view stereoscopic display. In the proposed method, feature points are extracted from the stereo image pair by detecting the comers and similarities between each pixel of the stereo image pair. These detected feature points, are then used to extract moving vectors between the stereo image pair and the epipolar line. Finally, the input stereo image pair is rectified by matching the extracted epipolar line between the stereo image pair in the horizontal direction. Based on some experiments done on the synthesis of the intermediate views by using the calibrated stereo image pairs through the proposed rectification algorithm and the uncalibrated ones for three kinds of stereo image pairs; 'Man', 'Face' and 'Car', it is found that PSNRs of the intermediate views reconstructed from the calibrated images improved by about 2.5${\sim}$3.26 dB than those of the uncalibrated ones.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2020.07a
• /
• pp.144-145
• /
• 2020

본 논문에서는 정밀한 3차원 복원 및 시점 합성을 위해 매칭 비용을 반복적으로 업데이트하는 Generalized Soft 3D Reconstruction (GenSoft3D) 알고리즘을 제안한다. 먼저 다시점 영상들과 카메라 자세정보가 주어지면 GenSoft3D는 볼륨 기반의 다시점 스테레오 매칭 알고리즘으로 시점별 초기 매칭 비용 볼륨 및 시차 맵을 계산한다. 그 후 정제 과정에서 각 시점은 모든 시차 맵을 이용하여 표면 확률 및 가시 확률을 계산한다. 표면 확률은 초기 매칭 비용 업데이트에 사용하며, 가시 확률은 폐색 영역의 정확한 시차를 계산하기 위해 사용된다. 해당 정제 과정을 일정 횟수 반복할 경우 시점별 고정밀의 시차 맵 획득이 가능하다. 또한 시차 맵의 정확도가 향상됨에 따라 정확한 시점 합성이 가능하다.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.34 no.12C
• /
• pp.1154-1166
• /
• 2009

In this paper, we propose algorithms for generating high quality virtual intermediate views on the baseline or out of baseline. In this proposed algorithm, depth information as well as 3D warping technique is used to generate the virtual views. The coordinate of real 3D image is calculated from the depth information and geometrical characteristics of camera and the calculated 3D coordinate is projected to the 2D plane at arbitrary camera position and results in 2D virtual view image. Through the experiments, we could show that the generated virtual view image on the baseline by the proposed algorithm has better PSNR at least by 0.5dB and we also could cover the occluded regions more efficiently for the generated virtual view image out of baseline by the proposed algorithm.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2016.11a
• /
• pp.89-91
• /
• 2016

본 논문에서는 실사 영상 기반으로 3D 영상을 생성하기 위하여 효율적으로 다시점 영상을 획득하는 시스템을 제안한다. 기존의 시스템은 대부분 다수의 카메라를 이용하여 다시점 영상을 획득하는 구조이다. 이 경우 각 카메라 간의 정합(calibration)을 수행해야 할 뿐만 아니라 스테레오 매칭을 통해 깊이 정보를 추출하는 과정이 필요하다. 제안하는 시스템에서는 카메라는 고정시킨 상태에서 촬영하고자 하는 객체를 턴테이블 위에 놓고 회전시키면서 촬영한다. 카메라는 Microsoft에서 출시한 컬러 정보와 깊이 정보를 동시에 얻을 수 있는 키넥트(Kinect) v2를 사용한다. 실험을 통하여 제안하는 시스템이 기존 시스템보다 다시점 영상을 효율적으로 생성하는 것을 확인하였다.