• The Journal of The Korea Institute of Intelligent Transport Systems
• /
• v.20 no.6
• /
• pp.313-330
• /
• 2021

In an autonomous driving system equipped with a camera and lidar, depth completion techniques enable dense depth estimation. In particular, using self-supervised learning it is possible to train the depth completion network even without ground truth. In actual autonomous driving, such depth completion should have very short latency as it is the input of other algorithms. So, rather than complicate the network structure to increase the accuracy like previous studies, this paper focuses on network latency. We design a U-Net type network with RegNet encoders optimized for GPU computation. Instead, this paper presents several techniques that can increase accuracy during the process of self-supervised learning. The proposed techniques increase the robustness to unreliable lidar inputs. Also, they improve the depth quality for edge and sky regions based on the semantic information extracted in advance. Our experiments confirm that our model is very lightweight (2.42 ms at 1280x480) but resistant to noise and has qualities close to the latest studies.

• Journal of Broadcast Engineering
• /
• v.27 no.3
• /
• pp.341-350
• /
• 2022

In this paper, we propose a deep learning based on-device augmented reality (AR) system in which multiple input images are used to implement the correct occlusion in a real environment. The proposed system is composed of three technical steps; camera pose estimation, depth estimation, and object augmentation. Each step employs various mobile frameworks to optimize the processing on the on-device environment. Firstly, in the camera pose estimation stage, the massive computation involved in feature extraction is parallelized using OpenCL which is the GPU parallelization framework. Next, in depth estimation, monocular and multiple image-based depth image inference is accelerated using the mobile deep learning framework, i.e. TensorFlow Lite. Finally, object augmentation and occlusion handling are performed on the OpenGL ES mobile graphics framework. The proposed augmented reality system is implemented as an application in the Android environment. We evaluate the performance of the proposed system in terms of augmentation accuracy and the processing time in the mobile as well as PC environments.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.34 no.11B
• /
• pp.1303-1310
• /
• 2009

Glassless 3D display requires multiple images taken from different viewpoints to show a scene. The simplest way to get multi-view image is using multiple camera that as number of views are requires. To do that, synchronize between cameras or compute and transmit lots of data comes critical problem. Thus, generating such a large number of viewpoint images effectively is emerging as a key technique in 3D video technology. Image-based view synthesis is an algorithm for generating various virtual viewpoint images using a limited number of views and depth maps. In this paper, because the virtual view image can be express as a transformed image from real view with some depth condition, we propose an algorithm to compute multi-view synthesis from two reference view images and their own depth-map by stepwise duplex forward mapping. And also, because the geometrical relationship between real view and virtual view is repetitively, we apply our algorithm into OpenGL Shading Language which is a programmable Graphic Process Unit that allow parallel processing to improve computation time. We demonstrate the effectiveness of our algorithm for fast view synthesis through a variety of experiments with real data.

• 한국HCI학회:학술대회논문집
• /
• 2008.02a
• /
• pp.433-438
• /
• 2008

본 논문은 비주얼 헐 오브젝트를 이용한 움직이는 오브젝트에 대한 실시간 영상기반 라이팅 기술을 제안한다. 본 논문에서는 특히 서로 다른 공간상의 조명 환경을 일치시키는 기술에 중점을 두고, 실시간으로 움직이는 오브젝트의 실시간 비디오 기반 재조명 기술로서 3가지 핵심 내용을 소개한다. 첫째는 비주얼 헐 데이터를 기반으로 기존에 벡터의 외적을 사용하던 방법을 개선하여 수식을 근사화시켜 연산량을 줄여서 고속으로 노말 벡터를 추출하는 방법이고, 둘째는 사용자 주변 조명 환경 정보를 효과적으로 샘플링하여 라이팅에 사용하는 점광원의 개수를 줄였으며, 세 번째는 CPU와 GPU의 연산량을 분배하여 효과적으로 병렬 고속 연산이 가능하도록 하였다. 종래의 영상기반 라이팅 기술이 정지된 환경맵 영상을 사용하거나 정지된 객체를 라이팅하였던 연구를 한 반면에 본 논문은 실시간에서 라이팅을 구현하기 위한 기술로서 고속 라이팅 연산을 위한 방법을 제시하고 있다. 본 연구의 결과를 이용하면 영상기반 라이팅 연구의 실제적이고도 폭넓은 작용이 가능할 것으로 사료되며 고화질의 콘텐츠 양산에도 기여할 것으로 사료된다.

• Journal of Biomedical Engineering Research
• /
• v.34 no.3
• /
• pp.135-140
• /
• 2013

We have developed an ultra wide-field of view Optical Coherence Tomography(OCT) which has capability to 2D and 3D views of cross-sectional structure of in vivo human retina. Conventional OCT has a limitation in visualizing the entire retina due to a reduced field of view. We designed an optical setup to significantly improve the lateral scanning range to be more than 20 mm. The entire human retinal structure in 2D and 3D was reported in this paper with the developed OCT system. Also, we empirically searched an optimized image size for real time visualization by analyzing variation of the frame rate with different lateral scan points. The size was concluded to be $1024{\times}2000{\times}300$ pixels which took 9 seconds for visualization.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2012.07a
• /
• pp.140-141
• /
• 2012

최근 영상분야의 키워드는 초고품질화, 초실감화, 스마트화로 대표될 수 있다. 그 중에서도 무안경식 3D는 초실감화를 이루기 위한 핵심응용분야 중 하나이다. 하지만 무안경식 3D 단말기가 성공적으로 보급되기 위해서는 연구되어야 할 분야가 여전히 존재한다. 그 중에서도 본 논문에서는 고화질의 무안경식 3D 스마트 콘텐츠 제작에 필요한 자동 스테레오 정합 기법을 제안하였다. 이전까지 연구된 변이지도 추출을 위한 알고리즘은 전역적 최적화 방법을 사용할 시 영상의 해상도와 깊이 정도에 따른 연산량의 증가로 많은 수행시간이 요구되었다. 또한 좌/우 영상의 intensity 정보만으로는 정확한 변이지도 추출이 어렵다는 한계점이 존재하였다. 이러한 이유로 본 논문에서는 스트림 영상에서 프레임 간의 정보를 이용하여 신뢰지도와 경계정보를 생성하였으며 belief propagation 스테레오 정합 방법을 이용하여 고화질의 정확한 변이지도를 추출하였다. 또한, 알고리즘의 연산량에 대한 문제를 해결하기 위한 고속화 방안으로, 최근 많은 연구가 이루어지고 있는 GPU(graphics processing units) 를 이용한 병렬처리를 연구하였다. 마지막으로 연구결과의 신뢰성을 향상하기 위하여 다양한 데이터를 이용한 실험을 통해 고화질의 영상정보를 고속으로 추출할 수 있음을 확인하였다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2016.04a
• /
• pp.189-191
• /
• 2016

실시간 물리 기반 3D 시뮬레이션에서 연산속도는 매우 중요한 요소이다. 객체의 움직임이나 변형과 같은 현상들은 복잡한 연산을 통해서 계산되기 때문에 일반적으로 시뮬레이션의 정확도와 연산속도는 반비례 관계에 있다. 현재 출시되고 있는 대부분의 게임에서는 물체의 움직임을 정확하게 표현하기보다 연산량을 줄이기 위해 물체의 움직임이나 변형을 비슷하게 표현하는데 중점을 두고 있다. 본 논문에서는 이러한 문제를 해결하기 위하여 OpenGL 4.3의 Compute shader를 사용하여 다이내믹 시뮬레이션의 연산 작업을 GPU 병렬처리로 처리하였다. Compute shader에서 파티클의 움직임을 계산하고 Shader storage buffer object에 저장하고 파티클들의 작업량을 적절한 Workgroup의 크기로 나누어 할당하여 최적의 처리속도를 제공하도록 구현하였다. Compute shader에서 파티클의 움직임을 표현하기 위해서 수치해법 중의 하나인 Euler method를 사용하였으며 실험 결과 파티클의 수가 4,194,304개일 때 CPU 방법에 비해 약 182배 빠른 연산속도 결과를 보였다. 추후 Compute shader를 활용하여 연산량이 많은 분야에 적용 가능할 수 있을 것으로 기대한다.

• Journal of The Institute of Information and Telecommunication Facilities Engineering
• /
• v.10 no.1
• /
• pp.7-13
• /
• 2011

Matrix multiplication is a fundamental operation of linear algebra and arises in many areas of science and engineering. This paper introduces an efficient parallel matrix multiplication scheme on N ${\times}$ N mesh-connected SIMD array processor, called multiple hierarchical SIMD architecture (HMSA). The architectural characteristic of HMSA is the hierarchically structured control units which consist of a global control unit, N local control units configured diagonally, and $N^2$ processing elements (PEs) arranged in an N ${\times}$ N array. PEs are communicating through local buses connecting four adjacent neighbor PEs in mesh-torus networks and global buses running across the rows and columns called horizontal buses and vertical buses, respectively. This architecture enables HMSA to have the features of diagonally indexed concurrent broadcast and the accessibility to either rows (row control mode) or columns (column control mode) of 2D array PEs alternately. An algorithmic mapping method is used for performance evaluation by mapping matrix multiplication on the proposed architecture. The asymptotic time complexities of them are evaluated and the result shows that paralle matrix multiplication on HMSA can provide significant performance improvement.

• Journal of IKEEE
• /
• v.20 no.3
• /
• pp.303-306
• /
• 2016

In this paper, we propose a method to accelerate convolutional neural network by utilizing a GPGPU. Convolutional neural network is a sort of the neural network learning features of images. Convolutional neural network is suitable for the image processing required to learn a lot of data such as images. The convolutional layer of the conventional CNN required a large number of multiplications and it is difficult to operate in the real-time on the embedded environment. In this paper, we reduce the number of multiplications through Winograd convolution operation and perform parallel processing of the convolution by utilizing SIMT-based GPGPU. The experiment was conducted using ModelSim and TestDrive, and the experimental results showed that the processing time was improved by about 17%, compared to the conventional convolution.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.66 no.1
• /
• pp.100-106
• /
• 2017

In this paper, we propose a parallel optimization method of Aho-Corasick (AC) algorithm and Parallel Failureless Aho-Corasick (PFAC) algorithm using Open Computing Language (OpenCL) on Field Programmable Gate Array (FPGA). The low throughput of string matching engine causes the performance degradation of network process. Recently, many researchers have studied the string matching engine using parallel computing. FPGA's vendors offer a parallel computing platform using OpenCL. In this paper, we apply the AC and PFAC algorithm on DE1-SoC board with Cyclone V FPGA, where the optimization that considers FPGA architecture is performed. Experiments are performed considering global id, local id, local memory, and loop unrolling optimizations using PFAC algorithm. The performance improvement using loop unrolling is 129 times greater than AC algorithm that not adopt loop unrolling. The performance improvements using loop unrolling are 1.1, 0.2, and 1.5 times greater than those using global id, local id, and local memory optimizations mentioned above.