• ETRI Journal
• /
• v.44 no.1
• /
• pp.24-37
• /
• 2022

Virtual view synthesis, which generates novel views similar to the characteristics of actually acquired images, is an essential technical component for delivering an immersive video with realistic binocular disparity and smooth motion parallax. This is typically achieved in sequence by warping the given images to the designated viewing position, blending warped images, and filling the remaining holes. When considering 6DoF use cases with huge motion, the warping method in patch unit is more preferable than other conventional methods running in pixel unit. Regarding the prior case, the quality of synthesized image is highly relevant to the means of blending. Based on such aspect, we proposed a novel blending architecture that exploits the similarity of the directions of rays and the distribution of depth values. By further employing the proposed method, results showed that more enhanced view was synthesized compared with the well-designed synthesizers used within moving picture expert group (MPEG-I). Moreover, we explained the GPU-based implementation synthesizing and rendering views in the level of real time by considering the applicability for immersive video service.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.38C no.10
• /
• pp.920-927
• /
• 2013

Chroma key technique which composes images by separating an object from its background in specific color has restrictions on color and space. Especially, unlike general chroma key technique, image composition for stereo 3D display requires natural image composition method in 3D space. The thesis attempted to compose images in 3D space using depth keying method which uses high resolution depth information. High resolution depth map was obtained through camera calibration between the DSLR and Kinect sensor. 3D mesh model was created by the high resolution depth information and mapped with RGB color value. Object was converted into point cloud type in 3D space after separating it from its background according to depth information. The image in which 3D virtual background and object are composed obtained and played stereo 3D images using a virtual camera.

• Journal of Broadcast Engineering
• /
• v.22 no.4
• /
• pp.429-437
• /
• 2017

2D-to-3D conversion technology has been studied over past decades and integrated to commercial 3D displays and 3DTVs. Generally, depth cues extracted from a static image is used for generating a depth map followed by DIBR (Depth Image Based Rendering) for producing a stereoscopic image. Further, motion is also an important cue for depth estimation and is estimated by block-based motion estimation, optical flow and so forth. This papers proposes a new method for motion depth generation using Motion History Image (MHI) and evaluates the feasiblity of the MHI utilization. In the experiments, the proposed method was performed on eight video clips with a variety of motion classes. From a qualitative test on motion depth maps as well as the comparison of the processing time, we validated the feasibility of the proposed method.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.53 no.2
• /
• pp.89-96
• /
• 2016

The High Efficiency Video Coding (MPEG-H HEVC/ITU-T H.265) is the newest video coding standard which has the quadtree-structured coding unit (CU). The quadtree-structure splits a CU adaptively, and its optimum CU depth can be determined by rate-distortion optimization. Such HEVC encoding requires very high computational complexity for CU depth decision. Motivated that the blob detection, which is a well-known algorithm in computer vision, detects keypoints in pictures and decision of CU depth needs to consider high frequency energy distribution, in this paper, we propose to utilize these keypoints for fast CU depth decision. Experimental results show that 20% encoding time can be saved with only slightly increasing BDBR by 0.45% on all intra case.

• Electronics and Telecommunications Trends
• /
• v.34 no.6
• /
• pp.156-163
• /
• 2019

As a primitive immersive video technology, a three degrees of freedom (3DoF) $360^{\circ}$ video can currently render viewport images that are dependent on the rotational movements of the viewer. However, rendering a flat $360^{\circ}$ video, that is supporting head rotations only, may generate visual discomfort especially when objects close to the viewer are rendered. 3DoF+ enables head movements for a seated person adding horizontal, vertical, and depth translations. The 3DoF+ $360^{\circ}$ video is positioned between 3DoF and six degrees of freedom, which can realize the motion parallax with relatively simple virtual reality software in head-mounted displays. This article introduces the standardization trends for the 3DoF+ video in the MPEG-I visual group.

• Journal of Broadcast Engineering
• /
• v.20 no.3
• /
• pp.368-378
• /
• 2015

3D conversion technology has been studied over past decades and integrated to commercial 3D displays and 3DTVs. The 3D conversion plays an important role in the augmented functionality of three-dimensional television (3DTV), because it can easily provide 3D contents. Generally, depth cues extracted from a static image is used for generating a depth map followed by DIBR (Depth Image Based Rendering) rendering for producing a stereoscopic image. However except some particular images, the existence of depth cues is rare so that the consistent quality of a depth map cannot be accordingly guaranteed. Therefore, it is imperative to make a 3D conversion method that produces satisfactory and consistent 3D for diverse video contents. From this viewpoint, this paper proposes a novel method with applicability to general types of image. For this, saliency as well as edge is utilized. To generate a depth map, geometric perspective, affinity model and binomic filter are used. In the experiments, the proposed method was performed on 24 video clips with a variety of contents. From a subjective test for 3D perception and visual fatigue, satisfactory and comfortable viewing of 3D contents was validated.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.7 no.11
• /
• pp.2839-2852
• /
• 2013

Context-awareness is an essential part of ubiquitous computing, and over the past decade video based activity recognition (VAR) has emerged as an important component to identify user's context for automatic service delivery in context-aware applications. The accuracy of VAR significantly depends on the performance of the employed human body segmentation algorithm. Previous human body segmentation algorithms often engage modeling of the human body that normally requires bulky amount of training data and cannot competently handle changes over time. Recently, active contours have emerged as a successful segmentation technique in still images. In this paper, an active contour model with the integration of Chan Vese (CV) energy and Bhattacharya distance functions are adapted for automatic human body segmentation using depth cameras for VAR. The proposed technique not only outperforms existing segmentation methods in normal scenarios but it is also more robust to noise. Moreover, it is unsupervised, i.e., no prior human body model is needed. The performance of the proposed segmentation technique is compared against conventional CV Active Contour (AC) model using a depth-camera and obtained much better performance over it.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.33 no.1C
• /
• pp.53-61
• /
• 2008

In this paper, we propose an algorithm for generation of multi-view video from conventional 2 dimensional video. Color and motion information of an object are used for segmentation and from the segmented objects, multi-view video is generated. Especially, color information is used to extract the boundary of an object that is barely extracted by using motion information. To classify the homogeneous regions with color, luminance and chrominance components are used. A pixel-based motion estimation with a measurement window is also performed to obtain motion information. Then, we combine the results from motion estimation and color segmentation and consequently we obtain a depth information by assigning motion intensity value to each segmented region. Finally, we generate multi-view video by applying rotation transformation method to 2 dimensional input images and the obtained depth information in each object. The experimental results show that the proposed algorithm outperforms comparing with conventional conversion methods.

• Journal of Broadcast Engineering
• /
• v.23 no.6
• /
• pp.886-895
• /
• 2018

In this paper, we introduce a technique to detect the video forgery by using the regularity that occurs in the video compression process. The proposed method uses the hierarchical regularity lost by the video double compression and the frame deletion. In order to extract such irregularities, the depth information of CU and TU, which are basic units of HEVC, is used. For improving performance, we make a depth map of CU and TU using local information, and then create input data by grouping them in GoP units. We made a decision whether or not the video is double-compressed and forged by using a general three-dimensional convolutional neural network. Experimental results show that it is more effective to detect whether or not the video is forged compared with the results using the existing machine learning algorithm.

• The Journal of the Korea Contents Association
• /
• v.12 no.2
• /
• pp.173-181
• /
• 2012

The education method using visual materials such as video contents is getting more attentions in the education field. This study researched how undergraduate students perceive the objective of education method utilizing video contents, what types of education using video contents they put a great importance on, and what kinds of correlations exist between the video contents method and the ability of discussing about a topic in essay-type examinations. With in-depth interviews with undergraduate students, the results showed that video contents was perceived as one of the attractive teaching methods increasing students' comprehension of lectures. Also, students evaluated that the video contents method could provide an opportunity to be exposed to the vivid scene of the journalistic environment and make them learn practical cases at second hand. Moreover, they perceived the video contents method as a very effective learning method improving their ability to discuss about a topic in essay-type exams.