• Journal of Broadcast Engineering
• /
• v.27 no.2
• /
• pp.174-184
• /
• 2022

3D human pose estimation models can be classified into a multi-view model and a single-view model. In general, the multi-view model shows superior pose estimation performance compared to the single-view model. In the case of the single-view model, the improvement of the 3D pose estimation performance requires a large amount of training data. However, it is not easy to obtain annotations for training 3D pose estimation models. To address this problem, we propose a method to generate pseudo ground-truths of multi-view human pose data from a multi-view model and exploit the resultant pseudo ground-truths to train a single-view model. In addition, we propose a multi-view consistency loss function that considers the consistency of poses estimated from multi-view images, showing that the proposed loss helps the effective training of single-view models. Experiments using Human3.6M and MPI-INF-3DHP datasets show that the proposed method is effective for training single-view 3D human pose estimation models.

• IEIE Transactions on Smart Processing and Computing
• /
• v.3 no.3
• /
• pp.118-127
• /
• 2014

This paper presents an efficient motion and disparity prediction method for multi-view video coding based on the high efficient video coding (HEVC) standard. The proposed method exploits inter-view candidates for effective prediction of the motion or disparity vector to be coded. The inter-view candidates include not only the motion vectors of adjacent views, but also global disparities across views. The motion vectors coded earlier in an adjacent view were found to be helpful in predicting the current motion vector to reduce the number of bits used in the motion vector information. In addition, the proposed disparity prediction using the global disparity method was found to be effective for interview predictions. A multi-view version based on HEVC was used to evaluate the proposed algorithm, and the proposed correspondence prediction method was implemented on a multi-view platform based on HEVC. The proposed algorithm yielded a coding gain of approximately 2.9% in a high efficiency configuration random access mode.

• Proceedings of the IEEK Conference
• /
• 2005.11a
• /
• pp.375-378
• /
• 2005

The multi-view video is a collection of multiple videos capturing the same scene at different viewpoints. The multi-view video can be used in various applications, including free viewpoint TV and three-dimensional TV. Since the data size of the multi-view video linearly increases as the number of cameras, it is necessary to compress multi-view video data for efficient storage and transmission. The multi-view video can be coded using the concept of the layered depth image (LDI). In this paper, we describe a procedure to generate LDI from the natural multi-view video and present a method to encode multi-view video using the concept of LDI.

• Journal of Korea Multimedia Society
• /
• v.3 no.1
• /
• pp.34-40
• /
• 2000

'[n this paper, we propose an efficient multi-view image coding algorithm which finds the optimal texture from a restricted number of multi-view image. The X-Y plane of the normalized object space is divided into the triangular patches. The depth of each node is determined by appling a block based disparity compensation method. Thereafter the texture of each patch is extracted by appling an affine transformation based disparity compensation method to the multi-view images. We reduced the number of images needed to determine the texture compared to traditional methods which use all the multi-view image in the texture extraction. The experimental results show that the SNR of images encoded by the proposed algorithm is better than that of images encoded by the traditional method by the approximately 0.2dB for the test sets of multi -view image called dragon, santa, city and kid. Image data recovered after encoding by the proposed method show a better visual results than after using traditional method.

• The Korean Journal of Applied Statistics
• /
• v.32 no.1
• /
• pp.41-68
• /
• 2019

Multi-view learning considers data from various viewpoints as well as attempts to integrate various information from data. Multi-view learning has been studied recently and has showed superior performance to a model learned from only a single view. With the introduction of deep learning techniques to a multi-view learning approach, it has showed good results in various fields such as image, text, voice, and video. In this study, we introduce how multi-view learning methods solve various problems faced in human behavior recognition, medical areas, information retrieval and facial expression recognition. In addition, we review data integration principles of multi-view learning methods by classifying traditional multi-view learning methods into data integration, classifiers integration, and representation integration. Finally, we examine how CNN, RNN, RBM, Autoencoder, and GAN, which are commonly used among various deep learning methods, are applied to multi-view learning algorithms. We categorize CNN and RNN-based learning methods as supervised learning, and RBM, Autoencoder, and GAN-based learning methods as unsupervised learning.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.11
• /
• pp.5546-5559
• /
• 2019

Facial action units (AUs) have recently drawn increased attention because they can be used to recognize facial expressions. A variety of methods have been designed for frontal-view AU detection, but few have been able to handle multi-view face images. In this paper we propose a method for multi-view facial AU detection using a fused multilayer, multi-task, and multi-label deep learning network. The network can complete two tasks: AU detection and facial view detection. AU detection is a multi-label problem and facial view detection is a single-label problem. A residual network and multilayer fusion are applied to obtain more representative features. Our method is effective and performs well. The F1 score on FERA 2017 is 13.1% higher than the baseline. The facial view recognition accuracy is 0.991. This shows that our multi-task, multi-label model could achieve good performance on the two tasks.

• Proceedings of the IEEK Conference
• /
• 2007.07a
• /
• pp.281-282
• /
• 2007

In this paper, we propose a method for digital matting using a multi-view camera system. In order to generate multi-view synthetic aperture images, we first move all images obtained from the multi-view camera according to their disparities. After we obtain corresponding trimaps by taking a variance of the synthetic aperture images, we convert the trimaps into multi-view alpha mattes. Experimental results show that the proposed scheme can create the composite images successfully by combining foreground objects with multi-view background images.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.4 no.6
• /
• pp.1222-1236
• /
• 2010

In this paper, we propose an improved distributed multi-view video coding method that is robust to illumination changes among different views. The use of view dependency is not effective for multi-view video because each view has different intrinsic and extrinsic camera parameters. In this paper, a modified distributed multi-view coding method is presented that applies illumination compensation when generating side information. The proposed encoder codes DC values of discrete cosine transform (DCT) coefficients separately by entropy coding. The proposed decoder can generate more accurate side information by using the transmitted DC coefficients to compensate for illumination changes. Furthermore, AC coefficients are coded with conventional entropy or channel coders depending on the frequency band. We found that the proposed algorithm is about 0.1~0.5 dB better than conventional algorithms.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.14 no.4
• /
• pp.29-37
• /
• 2014

Since multi-view video exists a number of camera color image and depth image, it has a huge of data. Thus, a new compression technique is indispensable for reducing this data. Recently, the effective compression encoding technique for multi-view video that used in layered depth image concepts is a remarkable. This method uses several view point of depth information and warping function, synthesizes multi-view color and depth image, becomes one data structure. In this paper we use actual distance for solving overlap in layered depth image that reduce required data for reconstructing in color-based transform. In experimental results, we confirmed high compression performance and good quality of reconstructed image.

• Journal of Broadcast Engineering
• /
• v.16 no.2
• /
• pp.258-273
• /
• 2011

This paper designs and implements a multi-view 3D video player system which is operated faster than existing video player systems. The structure for obtaining the near optimum speed in a multi-processor environment by parallelizing the component modules is proposed to process large volumes of multi-view image data at high speed. In order to use the concurrency of bottleneck, we designed image decoding, synthesis and rendering modules in a pipeline structure. For load balancing, the decoder module is divided into the unit of viewpoint, and the image synthesis module is geometrically divided based on synthesized images. As a result of this experiment, multi-view images were correctly synthesized and the 3D sense could be felt when watching the images on the multi-view autostereoscopic display. The proposed application processing structure could be used to process large volumes of multi-view image data at high speed, using the multi-processors to their maximum capacity.