• ETRI Journal
• /
• v.38 no.6
• /
• pp.1085-1094
• /
• 2016

In this work, we present interactive automultiscopic content with mobile surface haptics for multimodal interaction. Our system consists of a 40-view automultiscopic display and a tablet supporting surface haptics in an immersive room. Animated graphics are projected onto the walls of the room. The 40-view automultiscopic display is placed at the center of the front wall. The haptic tablet is installed at the mobile station to enable the user to interact with the tablet. The 40-view real-time rendering and multiplexing technology is applied by establishing virtual cameras in the convergence layout. Surface haptics rendering is synchronized with three-dimensional (3D) objects on the display for real-time haptic interaction. We conduct an experiment to evaluate user experiences of the proposed system. The results demonstrate that the system's multimodal interaction provides positive user experiences of immersion, control, user interface intuitiveness, and 3D effects.

• Journal of Broadcast Engineering
• /
• v.25 no.5
• /
• pp.655-664
• /
• 2020

Free viewpoint image synthesis technology is one of the important technologies in the MPEG-I (Immersive) standard. RVS (Reference View Synthesizer) developed by MPEG-I and in use in MPEG group is a DIBR (Depth Information-Based Rendering) program that generates an image at a virtual (intermediate) viewpoint from multiple viewpoints' inputs. RVS uses the mesh surface method based on computer graphics, and outperforms the pixel-based ones by 2.5dB or more compared to the previous pixel method. Even though its OpenGL version provides 10 times speed up over the non OpenGL based one, it still shows a non-real-time processing speed, i.e., 0.75 fps on the two 2k resolution input images. In this paper, we analyze the internal of RVS implementation and modify its structure, achieving 34 times speed up, therefore, real-time performance (22-26 fps), through the 3 key improvements: 1) the reuse of OpenGL buffers and texture objects 2) the parallelization of file I/O and OpenGL execution 3) the parallelization of GPU shader program and buffer transfer.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2022.06a
• /
• pp.1213-1215
• /
• 2022

몰입형 비디오 부호화를 위한 MIV(MPEG Immersive Video) 표준은 제한된 3D 공간의 다양한 위치의 뷰(view)들을 효율적으로 압축하여 사용자에게 임의의 위치 및 방향에 대한 6 자유도(6DoF)의 몰입감을 제공한다. MIV 의 참조 소프트웨어인 TMIV(Test Model for Immersive Video)에서는 복수의 뷰 간 중복되는 영역을 제거하여 전송할 화소수를 줄이기 때문에 복호화기에서 렌더링(rendering)을 위해서 각 화소의 점유(occupancy) 정보도 전송되어야 한다. TMIV 는 점유맵을 깊이(depth) 아틀라스(atlas)에 포함하여 압축 전송하고, 부호화 오류로 인한 점유 정보 손실을 방지하기 위해 깊이값 표현을 위한 동적 범위의 일부를 보호대역(guard band)으로 할당한다. 이 보호대역을 줄여서 더 넓은 깊이값의 동적 범위를 사용하면 렌더링 화질을 개선시킬 수 있다. 따라서, 본 논문에서는 현재 TMIV 의 점유 정보 오류 분석을 바탕으로 이를 보정하는 기법을 제시하고, 깊이 동적 범위 확장에 따른 부호화 성능을 분석한다. 제안기법은 기존의 TMIV 와 비교하여 평균 1.3%의 BD-rate 성능 향상을 보여준다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.5 no.2
• /
• pp.390-407
• /
• 2011

Three-dimensional (3D) video technologies are becoming increasingly popular, as it can provide users with high quality and immersive experiences. However, color inconsistency between the camera views is an urgent problem to be solved in multi-view imaging. In this paper, a perception-based color correction method for multi-view images is proposed. In the proposed method, human visual sensitivity (VS) and visual attention (VA) models are incorporated into the correction process. Firstly, the VS property is used to reduce the computational complexity by removing these visual insensitive regions. Secondly, the VA property is used to improve the perceptual quality of local VA regions by performing VA-dependent color correction. Experimental results show that compared with other color correction methods, the proposed method can greatly promote the perceptual quality of local VA regions greatly and reduce the computational complexity, and obtain higher coding performance.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.37 no.6A
• /
• pp.497-502
• /
• 2012

In this paper, we present an immersive videoconferencing system that enables gaze correction between users in the internet protocol TV (IPTV) environment. The proposed system synthesizes the gaze corrected images using the depth estimation and the virtual view synthesis algorithms as one of the most important techniques of 3D video system. The conventional processes, however, causes several problems, especially temporal inconsistency of a depth video. This problem leads to flickering artifacts discomforting viewers. Therefore, in order to reduce the temporal inconsistency problem, we exploit the joint bilateral filter which is extended to the temporal domain. In addition, we apply an outlier reduction operation in the temporal domain. From experimental results, we have verified that the proposed system is sufficient to generate the natural gaze-corrected image and realize immersive videoconferencing.

• The Journal of the Convergence on Culture Technology
• /
• v.8 no.3
• /
• pp.313-321
• /
• 2022

As 5G is getting developed, people are getting interested in immersive content. Some predicts that immersive content may be implemented in real life such as holograms, which were only possible in movies. Holograms, which has been studied for a long time since Dennis Gabor published the basic theory in 1948, are constantly developing in a new direction with digital technology. It is developing from a traditional optical hologram, which is produced by recording the interference pattern of light to a computer generated hologram (CGH) and a digital hologram printer. In order to produce a hologram using a digital hologram printer, holographic element (Hogel) image must first be created using multi-view images. There are a method of directly photographing an actual image and a method of modeling an object using 3D graphic production tool and rendering the motion of a virtual camera to acquire a series of multi-view images. In this paper, we propose a new method of getting image, which is one of the visual effect, VFX, producing multi-view images using chroma key composition. We shoot on the green screen of actual object, suggest the overall workflow of composition with 3D computer graphic(CG) and explain the role of each step. We expected that it will be helpful in researching a new method of image acquisition in the future if all or part of the proposed workflow to be applied.

• Cartoon and Animation Studies
• /
• s.17
• /
• pp.37-51
• /
• 2009

Immersion is the key factor in determining success or failure for fictional narrative animations. It does matter to not only for the socio-economic achievements like box-office success but to the asthetical achievement, immersion is very important prerequisite character that are raised in terms of research needs. Fictional narrative animation shares the audience's immersive method with the narrative content and Virtual Reality, since animation has the character which narrates itself with the visual image format (including sound) and added the fact that 3D format which emphasizes realism compared with the recent 2D animation has been ubiquitous phenomenon. In other words, the artistic for called 'animation' is located between the view point that text has the immersive point intrinsic and oppositely, specific function of the certain media which stays extrinsic of the text enforce the participants into the immersion. In any case, the subject who experience immersion is the audience, so the most useful theory to research the phenomenon 'immersion' is Constructivism cognitive narrotologic approach which lies the peader-spectator as the centric notion and is more useful than text analysis or research on the visual effects. In this study, research and review about the studies on the audience's immersive experience would be preceded particularly aroud the constructivism theories, and examine the uniqueness and naures of the animation which makes the audience immersive.

• Journal of Broadcast Engineering
• /
• v.24 no.5
• /
• pp.743-754
• /
• 2019

System for three degrees of freedom plus (3DoF+) and 6DoF requires multi-view high resolution 360 video transmission to provide user viewport adaptive 360 video streaming. In this paper, we implement 3DoF+ 360 video compression system which removes the redundancy between multi-view videos and merges the residual into one video to provide high quality 360 video corresponding to an user's head movement efficiently. Implementations about 3D warping based redundancy removal method between 3DoF+ 360 videos and residual extraction and merger are explained in this paper. With the proposed system, 20.14% of BD-rate reduction in maximum is shown compared to traditional high-efficiency video coding (HEVC) based system.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2015.07a
• /
• pp.524-526
• /
• 2015

본 논문은 자유 시점조절과 깊이감 조절이 가능한 단일렌즈 양안시 3D 내시경과 Head-mounted display 를 사용한 새로운 로봇수술 시스템을 제안한다. 최근 들어서 여러 양안식 입체영상장치가 의료 영역에 적용되고 있다. 그러나 3D 의료 장비들은 zooming 과 자유로운 시점조절, 그리고 접사를 하는 데에 한계가 존재한다. 입체영상 장비에서 필연적으로 발생하는 이러한 문제점들은 3D 영화를 찍기 위해서 사용되는 것과 같은 2 대의 카메라와 2 개의 렌즈를 사용하는 데에서 원인을 찾을 수 있다. 이러한 문제점들은 Da Vinci 로봇 수술 시스템과 같은 가장 최신의 시스템에서도 해결되지 못하였다. 본 논문에서 제안하는 새로운 시스템은 지금까지 제시된 문제점들을 해결하고, 현재 존재하는 로봇 수술 시스템에 몰입 입체영상 수술이나 증강 현실 수술을 가능하게 하기 위한 목적으로 제안되었다.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2018.11a
• /
• pp.128-130
• /
• 2018

이 논문은 3D 입체 음향에 대한 좌표 취득 및 표출 방법을 제안한다. 3차원 좌표를 추정하기 위해 다시점 영상의 색상 영상을 통해 중간 시점의 깊이 영상을 생성한다. 깊이 영상 내에서 객체에 대한 이미지 좌표와 깊이 정보를 이용하여 3차원 좌표를 취득한다. 실험 결과를 통해 깊이 영상에서 도출된 객체의 좌표를 음원의 좌표로 설정하여 3D 입체 음향 표출의 효율을 높일 수 있음을 확인한다.