• ETRI Journal
• /
• v.44 no.6
• /
• pp.1034-1046
• /
• 2022

The learning-based multiview stereo (MVS) methods for three-dimensional (3D) reconstruction generally use 3D volumes for depth inference. The quality of the reconstructed depth maps and the corresponding point clouds is directly influenced by the spatial resolution of the 3D volume. Consequently, these methods produce point clouds with sparse local regions because of the lack of the memory required to encode a high volume of information. Here, we apply the atrous spatial pyramid pooling (ASPP) module in MVS methods to obtain dense feature maps with multiscale, long-range, contextual information using high receptive fields. For a given 3D volume with the same spatial resolution as that in the MVS methods, the dense feature maps from the ASPP module encoded with superior information can produce dense point clouds without a high memory footprint. Furthermore, we propose a 3D loss for training the MVS networks, which improves the predicted depth values by 24.44%. The ASPP module provides state-of-the-art qualitative results by constructing relatively dense point clouds, which improves the DTU MVS dataset benchmarks by 2.25% compared with those achieved in the previous MVS methods.

• Journal of Broadcast Engineering
• /
• v.27 no.4
• /
• pp.487-498
• /
• 2022

Depth map is an image that contains distance information in 3D space on a 2D plane and is used in various 3D vision tasks. Many existing depth estimation studies mainly use narrow FoV images, in which a significant portion of the entire scene is lost. In this paper, we propose a technique for generating 360° omnidirectional RGBD images from a sparse set of narrow FoV images. The proposed generative adversarial network based image generation model estimates the relative FoV for the entire panoramic image from a small number of non-overlapping images and produces a 360° RGB and depth image simultaneously. In addition, it shows improved performance by configuring a network reflecting the spherical characteristics of the 360° image.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2020.07a
• /
• pp.286-287
• /
• 2020

Depth map is the most common way of expressing 3D space in immersive media. In this paper, we propose a post-processing method to improve the quality of depth map. In proposed method, a depth map is divided into segments, and the plane of each segment estimated using RANSAC. In order to increase the accuracy of the RANSAC process, we apply matching reliability of each pixel in depth map as a weighting factor.

• Smart Media Journal
• /
• v.1 no.3
• /
• pp.15-21
• /
• 2012

Depth information is necessary for various three dimensional contents generation. Depth acquisition techniques can be categorized broadly into two approaches: active, passive depth sensors depending on how to obtain depth information. In this paper, we take a look at several ways of depth acquirement. We present not only depth acquisition methods using discussed ways, but also hybrid methods which combine both approaches to compensate for drawbacks of each approach. Furthermore, we introduce several matching cost functions and post-processing techniques to enhance the temporal consistency and reduce flickering artifacts and discomforts of users caused by inaccurate depth estimation in 3D video.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.25 no.1
• /
• pp.78-84
• /
• 2015

Depth estimation is often required for robot vision, 3D modeling, and motion control. Previous method is based on the focus measures which are calculated for a series of image by a single camera at different distance between and object. This method, however, has disadvantage of taking a long time for calculating the focus measure since the mask operation is performed for every pixel in the image. In this paper, we estimates the depth by using the focus measure of the boundary pixels located between the objects in order to minimize the depth estimate time. To detect the boundary of an object consisting of a straight line and a circle, we use the Hough transform and estimate the depth by using the focus measure. We performed various experiments for PCB images and obtained more effective depth estimation results than previous ones.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.3
• /
• pp.1189-1204
• /
• 2018

Detecting and capturing 3D human structures from the intensity-based image sequences is an inherently arguable problem, which attracted attention of several researchers especially in real-time activity recognition (Real-AR). These Real-AR systems have been significantly enhanced by using depth intensity sensors that gives maximum information, in spite of the fact that conventional Real-AR systems are using RGB video sensors. This study proposed a depth-based routine-logging Real-AR system to identify the daily human activity routines and to make these surroundings an intelligent living space. Our real-time routine-logging Real-AR system is categorized into two categories. The data collection with the use of a depth camera, feature extraction based on joint information and training/recognition of each activity. In-addition, the recognition mechanism locates, and pinpoints the learned activities and induces routine-logs. The evaluation applied on the depth datasets (self-annotated and MSRAction3D datasets) demonstrated that proposed system can achieve better recognition rates and robust as compare to state-of-the-art methods. Our Real-AR should be feasibly accessible and permanently used in behavior monitoring applications, humanoid-robot systems and e-medical therapy systems.

• International conference on construction engineering and project management
• /
• 2022.06a
• /
• pp.328-335
• /
• 2022

With the advance of robot capabilities and functionalities, construction robots assisting construction workers have been increasingly deployed on construction sites to improve safety, efficiency and productivity. For close-proximity human-robot collaboration in construction sites, robots need to be aware of the context, especially construction worker's behavior, in real-time to avoid collision with workers. To recognize human behavior, most previous studies obtained 3D human poses using a single camera or an RGB-depth (RGB-D) camera. However, single-camera detection has limitations such as occlusions, detection failure, and sensor malfunction, and an RGB-D camera may suffer from interference from lighting conditions and surface material. To address these issues, this study proposes a novel method of 3D human pose estimation by extracting 2D location of each joint from multiple images captured at the same time from different viewpoints, fusing each joint's 2D locations, and estimating the 3D joint location. For higher accuracy, the probabilistic representation is used to extract the 2D location of the joints, considering each joint location extracted from images as a noisy partial observation. Then, this study estimates the 3D human pose by fusing the probabilistic 2D joint locations to maximize the likelihood. The proposed method was evaluated in both simulation and laboratory settings, and the results demonstrated the accuracy of estimation and the feasibility in practice. This study contributes to ensuring human safety in close-proximity human-robot collaboration by providing a novel method of 3D human pose estimation.

• The KIPS Transactions:PartB
• /
• v.12B no.3 s.99
• /
• pp.247-256
• /
• 2005

One of the classic research problems in computer vision is that of stereo, i.e., the reconstruction of three dimensional shape from two or more images. This paper deals with the problem of extracting depth information of non-rigid dynamic 3D scenes from general 2D video sequences taken by monocular camera, such as movies, documentaries, and dramas. Depth of the blocks are extracted from the resultant block motions throughout following two steps: (i) calculation of global parameters concerned with camera translations and focal length using the locations of blocks and their motions, (ii) calculation of each block depth relative to average image depth using the global parameters and the location of the block and its motion, Both singular and non-singular cases are experimented with various video sequences. The resultant relative depths and ego-motion object shapes are virtually identical to human vision.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.3
• /
• pp.1121-1141
• /
• 2020

We propose a robust visual object tracking algorithm fusing a convolutional neural network tracker trained offline from a large number of video repositories and a color histogram based tracker to track objects for mixing immersive audio. Our algorithm addresses the problem of occlusion and large movements of the CNN based GOTURN generic object tracker. The key idea is the offline training of a binary classifier with the color histogram similarity values estimated via both trackers used in this method to opt appropriate tracker for target tracking and update both trackers with the predicted bounding box position of the target to continue tracking. Furthermore, a histogram similarity constraint is applied before updating the trackers to maximize the tracking accuracy. Finally, we compute the depth(z) of the target object by one of the prominent unsupervised monocular depth estimation algorithms to ensure the necessary 3D position of the tracked object to mix the immersive audio into that object. Our proposed algorithm demonstrates about 2% improved accuracy over the outperforming GOTURN algorithm in the existing VOT2014 tracking benchmark. Additionally, our tracker also works well to track multiple objects utilizing the concept of single object tracker but no demonstrations on any MOT benchmark.

• Journal of the Semiconductor & Display Technology
• /
• v.13 no.2
• /
• pp.23-28
• /
• 2014

For the reconstruction of three-dimensional (3D) shape of microscopic objects through shape from focus (SFF) methods, usually a single focus measure operator is employed. However, it is difficult to compute accurate depth map using a single focus measure due to different textures, light conditions and arbitrary object surfaces. Moreover, real images with diverse types of illuminations and contrasts lead to the erroneous depth map estimation through a single focus measure. In order to get better focus measurements and depth map, we have combined focus measure operators by using genetic algorithm. The resultant focus measure is obtained by weighted sum of the output of various focus measure operators. Optimal weights are obtained using genetic algorithm. Finally, depth map is obtained from the refined focus volume. The performance of the developed method is then evaluated by using both the synthetic and real world image sequences. The experimental results show that the proposed method is more effective in computing accurate depth maps as compared to the existing SFF methods.