• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2018.11a
• /
• pp.141-142
• /
• 2018

360 비디오는 VR 미디어의 확산과 함께 몰입형 미디어로 주목 받고 있으며, JVET(Joint Video Experts Team)에서는 post-HEVC 로 진행중인 VVC(Versatile Video Coding) 표준화에 360 비디오 부호화도 함께 포함하고 있다. 현재 JVET 에서는 360 비디오를 부호화 하기 위한 다양한 구(sphere) 영상의 2D 투영기법이 고려되고 있다. 이러한 2D 투영에서는 구 영상의 화소 샘플이 2D 영상에 비 균일하게 매핑되는 변환 왜곡이 발생하며, 이는 360 비디오의 부호화 효율을 저하시키는 원인이 된다. 본 논문에서는 CMP 의 개선된 투영기법인 기존의 EAC(Equi-Angular Cubemap)와 HEC(Hybrid Equiangular Cubemap)를 소개하고, 이를 바탕으로 HEC 의 확장 변환 기법을 제시하여 객관적/주관적 부호화 성능을 확인한다.

• Journal of Korea Game Society
• /
• v.17 no.6
• /
• pp.153-164
• /
• 2017

In spherical panoramic images, SURF, a feature description method for planar patterns, does not work correctly due to heavy spherical distortion. Since a plane pattern is distorted in a spherical image, the pattern search and replacement in a spherical panoramic image should be treated differently from the case of the planar image. This paper proposes a planar texture replacement method, which transforms a spherical panoramic image into a cubemap panoramic image, searches a pattern using SURF, replaces a plane pattern, and then converts it into a spherical panoramic image.

• Journal of Korea Game Society
• /
• v.20 no.6
• /
• pp.43-52
• /
• 2020

This paper presents a fast pattern tracking method using location prediction in cubemap video for 360-degree VR. A spherical cubemap frame has six face textures and searching a pattern is much slower than a flat image. To overcome the limitation, we propose a method of predicting the location of target pattern using Kalman filter and reducing the search area by considering only textures of predicted location. The experimental results showed that the proposed system is much faster than the previous method of searching all six faces and also gives accurate pattern tracking performance.

• Journal of Broadcast Engineering
• /
• v.22 no.1
• /
• pp.136-139
• /
• 2017

Recently, many companies and consumers has shown a lot of interest toward VR(Virtual Reality), so many VR devices such as HMD(Head mounted Display) and 360 degree VR camera are released on the market. Current encoded 360 degree VR video uses the codec which originally made for the conventional 2D video. Therefore, the compression efficiency isn't optimized because the en/decoder does not consider the characteristics of the 360 degree VR video. In this paper, we propose a method to improve the compression efficiency by using the reference frame which compensates for the distortions caused by characteristics the 360 degree VR video. Applying the proposed method we were able to increase the compression efficiency by providing better prediction.

• Journal of Korea Game Society
• /
• v.18 no.5
• /
• pp.5-14
• /
• 2018

A lot of research has been done on image recognition technique for planar images and the performance has also been improved. However, it is difficult to recognize objects in spherical panoramic images or images in special form which are given in various environments because of the spherical distortion given in different form from the planar case. In this paper, we show that the neural network recognition approach can be used for object recognition in spherical image and suggest a method of using cubemap transform in order to increase recognition accuracy in spherical image.

• Journal of the Korea Computer Graphics Society
• /
• v.23 no.3
• /
• pp.65-75
• /
• 2017

As a 360-degree image carries information of all directions, it often has too much information. Moreover, in order to investigate a 360-degree image on a 2D display, a user has to either click and drag the image with a mouse, or project it to a 2D panorama image, which inevitably introduces severe distortions. In consequence, investigating a 360-degree image and finding an object of interest in such a 360-degree image could be a tedious task. To resolve this issue, this paper proposes a method to find a region of interest and produces a 2D naturally looking image from a given 360-degree image that best matches a description given by a user in a natural language sentence. Our method also considers photo composition so that the resulting image is aesthetically pleasing. Our method first converts a 360-degree image to a 2D cubemap. As objects in a 360-degree image may appear distorted or split into multiple pieces in a typical cubemap, leading to failure of detection of such objects, we introduce a modified cubemap. Then our method applies a Long Short Term Memory (LSTM) network based object detection method to find a region of interest with a given natural language sentence. Finally, our method produces an image that contains the detected region, and also has aesthetically pleasing composition.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2018.06a
• /
• pp.200-201
• /
• 2018

In the workflow of 360 video coding of JVET (Joint Video Experts Team), firstly the 360 videos are projected onto the 2D plane with diverse projection formats, such as Equi-Rectangular Projection (ERP), Cubemap Projection (CMP), Rotated Sphere Projection (RSP), etc. The projection format of RSP has inactive regions in the converted 2D plane. The inactive regions may cause visual artifact as well as the reduction of the coding efficiency due to discontinuity at boundaries between active and inactive regions. In this paper, to overcome these problems, the inactive regions are padded by using two types of adjacent pixels. Then padded regions of RSP are blended with inactive regions padded by proposed method. The experimental results demonstrate that, in terms of end-to-end WS-PSNR-NN, the proposed method achieves 0.1% BD-rate reduction. In addition, the visual artifacts along the borders between discontinuous faces are noticeably reduced.

• Journal of Broadcast Engineering
• /
• v.22 no.3
• /
• pp.401-404
• /
• 2017

The 360 VR video has a format of a stereoscopic shape such as an isometric shape or a cubic shape or a cubic shape. Although these formats have different characteristics, they have in common that the resolution is higher than that of a normal 2D video. Therefore, it takes much longer time to perform coding/decoding on 360 VR video than 2D Video, so parallel processing techniques are essential when it comes to coding 360 VR video. HEVC, the state of art 2D video codec, uses Wavefront Parallel Processing (WPP) technology as a standard for parallelization. This technique is optimized for 2D videos and does not show optimal performance when used in 3D videos. Therefore, a suitable method for WPP is required for 3D video. In this paper, we propose WPP coding/decoding method which improves WPP performance on cube map format 3D video. The experiment was applied to the HEVC reference software HM 12.0. The experimental results show that there is no significant loss of PSNR compared with the existing WPP, and the coding complexity of 15% to 20% is further reduced. The proposed method is expected to be included in the future 3D VR video codecs.