• Journal of Broadcast Engineering
• /
• v.27 no.3
• /
• pp.341-350
• /
• 2022

In this paper, we propose a deep learning based on-device augmented reality (AR) system in which multiple input images are used to implement the correct occlusion in a real environment. The proposed system is composed of three technical steps; camera pose estimation, depth estimation, and object augmentation. Each step employs various mobile frameworks to optimize the processing on the on-device environment. Firstly, in the camera pose estimation stage, the massive computation involved in feature extraction is parallelized using OpenCL which is the GPU parallelization framework. Next, in depth estimation, monocular and multiple image-based depth image inference is accelerated using the mobile deep learning framework, i.e. TensorFlow Lite. Finally, object augmentation and occlusion handling are performed on the OpenGL ES mobile graphics framework. The proposed augmented reality system is implemented as an application in the Android environment. We evaluate the performance of the proposed system in terms of augmentation accuracy and the processing time in the mobile as well as PC environments.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2015.10a
• /
• pp.699-702
• /
• 2015

다중의 영상을 이용하여 하나의 파노라마 영상을 제작하는 기법은 컴퓨터 비전, 컴퓨터 그래픽스 등과 같은 여러 분야에서 널리 연구되고 있다. 파노라마 영상은 하나의 카메라에서 얻을 수 있는 영상의 한계, 즉 예를 들어 화각, 화질, 정보량 등의 한계를 극복할 수 있는 좋은 방법으로서 가상현실, 로봇비전 등과 같이 광각의 영상이 요구되는 다양한 분야에서 응용될 수 있다. 파노라마 영상은 단일 영상과 비교하여 보다 큰 몰입감을 제공한다는 점에서 큰 의미를 갖는다. 현재 다양한 파노라마 영상 제작 기법들이 존재하지만, 대부분의 기법들이 공통적으로 파노라마 영상을 구성할 때 각 영상에 존재하는 특징점 및 대응점을 검출하는 방식을 사용하고 있다. 본 논문에서 사용한 SURF(Speeded Up Robust Features) 알고리즘은 영상의 특징점을 검출할 때 영상의 흑백정보와 지역 공간 정보를 활용하는데, 영상의 크기 변화와 시점 검출에 강하며 SIFT(Scale Invariant Features Transform) 알고리즘에 비해 속도가 빠르다는 장점이 있어서 널리 사용되고 있다. 본 논문에서는 두 영상 사이 또는 하나의 영상과 여러 영상 사이에 대응되는 매칭을 계산하여 파노라마영상을 생성하는 처리 방법을 구현하고 기술하였다.

• Journal of Korea Game Society
• /
• v.15 no.5
• /
• pp.99-108
• /
• 2015

To identify the location of the object and improve the realism in 3D game, shadow mapping is widely used to compute the depth values of vertices in view of the light position. Since the depth value of the shadow map is calculated by the world coordinate, the depth values of the static object don't need to be updated. In this paper, (1) in order to improve the rendering speed, using multiple render targets the depth values of static objects stored only once are separated from those of dynamic objects stored each time. And (2) in order to improve the shadow quality in the quarter view 3D game, the position of the light is located close to dynamic objects traveled along the camera each time. The effectiveness of the proposed method is verified by the experiments according to the different static and dynamics object configuration in 3D game.

• Journal of Broadcast Engineering
• /
• v.19 no.5
• /
• pp.631-639
• /
• 2014

Since digital broadcasting service has been begun, user's requirements have been increased for personalized broadcasting system that users can watch the part of programs which they want to watch in anytime. In personalized broadcasting system, metadata is important for searching program information based on user's input data. In this paper, an object-oriented media service metadata management module is implemented. Metadata used in this system is described by extending TV-Anytime specifications which is the International standard for personalized media service. For this, extended segments information is provided including object information which do not existing in previous personalized broadcasting standard. This system can provide better satisfaction to users by using object segmentation information.

• KIPS Transactions on Software and Data Engineering
• /
• v.2 no.7
• /
• pp.495-502
• /
• 2013

Motion estimation (ME) that limits the performance of image quality and encoding speed has been developed to reduce temporal redundancy in video sequences and plays an important role in digital video compression. But it is computational demanding part of the encoder. Multi-view video is obtained by capturing one three-dimensional scene with many cameras at different positions. ME for Multi-view video requires high computational complexity. To reduce computational complexity and maintain the image quality, a fast motion estimation method is proposed in this paper. The proposed method uses a hierarchical search strategy. This strategy method consists of modified diamond search patten, multi gird diamond search pattern, and raster search pattern. These search patterns place search points symmetrically and evenly that can cover the overall search area not to fall into the local minimum or exploits the characteristics of the distribution of motion vectors to place the search points. Experiment results show that the speedup improvement of the proposed method over TZ search method (JMVC) can be up to 1.2 ~3 times faster while maintaining similar video quality and bit rates.

• Journal of Korea Multimedia Society
• /
• v.17 no.3
• /
• pp.333-341
• /
• 2014

Multi-view video is obtained by capturing one three-dimensional scene with many cameras at different positions. Multi-view video coding requires high computational complexity. To reduce computational complexity and maintain the image quality, an effective early termination motion estimation method is proposed in this paper. The proposed method exploiting the characteristic of motion vector distribution uses a hierarchical search strategy. This strategy method consists of multi-grid square search pattern, modified diamond search pattern, small diamond search pattern and raster search pattern. Experiment results show that the speedup improvement of the proposed method over TZ search method and FS(Full Search) method JMVC (Joint Multiview Video Coding) can be up to 1.7~4.5 times and 90 times faster respectively while maintaining similar video quality and bit rates.

• Cartoon and Animation Studies
• /
• s.33
• /
• pp.31-53
• /
• 2013

In the early twentieth century, history of animation began by modern artists, they produced various experimental images with the newly invented film and cameras. Artists in the field of movie, photography, paintings and others manipulated images in motion. But as some animated movies won industrial success and popularity, they became the trend but experimental style of early animation preserved by so-called non-mainstreamers or experimental animators, counteracting commercialism. Disney animation also followed the trend by applying realistic Hollywood film style, the worse critics placed a low value on the animation and it tarnished the image, although it was profitable investment from a business standpoint. To make images realistic, they opened a drawing class that animators developed skills to imitate motions and forms from subjects in real life. Also some techniques and gizmos were used to mimic and simulate three dimensional objects and spaces, multiplane camera and compositing 3D CG images with 2D drawings. Moreover, they brought animation stories from fairly tales or folk tales, and Walt's personal interest in live-action movies, they applied Hollywood-film-like narratives and realistic visual, and harsh criticism ensued. On the surface early disney animations' potential seems to be weakened, but in reality it still exists by simplifying and exaggerating forms and color as modern arts. Disney animation employs concepts of the modernism paintings such as simplified shapes and colors to a character design, when their characters are placed together in a scene, that visual elements cause mental reaction. This modification gives a new internal experience to audiences. As conceptual colors in abstract paintings make images appeared to be flat, coloring characters with no shading make them look flat and comparing to them, background images are also appeared to be flat. On top of that, multi-perspective at background images recalls modernist paintings. This essay goes in details with the animation pioneers' works and how Disney animation developed its techniques to emulate real life and analyses color schemes, forms, and spaces in Disney animation compared with modern artists' works, in that the visual language of Disney animation reminds of impression from abstract paintings in the beginning of the twentieth centuries.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.51 no.4
• /
• pp.144-159
• /
• 2014

Techniques for making a single panoramic image using multiple pictures are widely studied in many areas such as computer vision, computer graphics, etc. The panorama image can be applied to various fields like virtual reality, robot vision areas which require wide-angled shots as an useful way to overcome the limitations such as picture-angle, resolutions, and internal informations of an image taken from a single camera. It is so much meaningful in a point that a panoramic image usually provides better immersion feeling than a plain image. Although there are many ways to build a panoramic image, most of them are using the way of extracting feature points and matching points of each images for making a single panoramic image. In addition, those methods use the RANSAC(RANdom SAmple Consensus) algorithm with matching points and the Homography matrix to transform the image. The SURF(Speeded Up Robust Features) algorithm which is used in this paper to extract featuring points uses an image's black and white informations and local spatial informations. The SURF is widely being used since it is very much robust at detecting image's size, view-point changes, and additionally, faster than the SIFT(Scale Invariant Features Transform) algorithm. The SURF has a shortcoming of making an error which results in decreasing the RANSAC algorithm's performance speed when extracting image's feature points. As a result, this may increase the CPU usage occupation rate. The error of detecting matching points may role as a critical reason for disqualifying panoramic image's accuracy and lucidity. In this paper, in order to minimize errors of extracting matching points, we used $3{\times}3$ region's RGB pixel values around the matching points' coordinates to perform intermediate filtering process for removing wrong matching points. We have also presented analysis and evaluation results relating to enhanced working speed for producing a panorama image, CPU usage rate, extracted matching points' decreasing rate and accuracy.