• The Journal of Korea Robotics Society
• /
• v.15 no.3
• /
• pp.293-300
• /
• 2020

The purpose of our sensor system is to transparentize the large hydraulic manipulators of a six-ton dual arm excavator from the operator camera view. Almost 40% of the camera view is blocked by the manipulators. In other words, the operator loses 40% of visual information which might be useful for many manipulator control scenarios such as clearing debris on a disaster site. The proposed method is based on a 3D reconstruction technology. By overlaying the camera image from front top of the cabin with the point cloud data from RGB-D (red, green, blue and depth) cameras placed at the outer side of each manipulator, the manipulator-free camera image can be obtained. Two additional algorithms are proposed to further enhance the productivity of dual arm excavators. First, a color correction algorithm is proposed to cope with the different color distribution of the RGB and RGB-D sensors used on the system. Also, the edge overlay algorithm is proposed. Although the manipulators often limit the operator's view, the visual feedback of the manipulator's configurations or states may be useful to the operator. Thus, the overlay algorithm is proposed to show the edge of the manipulators on the camera image. The experimental results show that the proposed transparentization algorithm helps the operator get information about the environment and objects around the excavator.

• ETRI Journal
• /
• v.34 no.3
• /
• pp.466-469
• /
• 2012

In this letter, we propose a dual autostereoscopic display platform employing a natural interaction method, which will be useful for sharing visual data with users. To provide 3D visualization of a model to users who collaborate with each other, a beamsplitter is used with a pair of autostereoscopic displays, providing a visual illusion of a floating 3D image. To interact with the virtual object, we track the user's hands with a depth camera. The gesture recognition technique we use operates without any initialization process, such as specific poses or gestures, and supports several commands to control virtual objects by gesture recognition. Experiment results show that our system performs well in visualizing 3D models in real-time and handling them under unconstrained conditions, such as complicated backgrounds or a user wearing short sleeves.

• IEIE Transactions on Smart Processing and Computing
• /
• v.4 no.4
• /
• pp.224-230
• /
• 2015

This manuscript presents a real-time solution for 3D human body reconstruction with multiple RGB-D cameras. The proposed system uses four consumer RGB/Depth (RGB-D) cameras, each located at approximately $90^{\circ}$ from the next camera around a freely moving human body. A single mesh is constructed from the captured point clouds by iteratively removing the estimated overlapping regions from the boundary. A cell-based mesh construction algorithm is developed, recovering the 3D shape from various conditions, considering the direction of the camera and the mesh boundary. The proposed algorithm also allows problematic holes and/or occluded regions to be recovered from another view. Finally, calibrated RGB data is merged with the constructed mesh so it can be viewed from an arbitrary direction. The proposed algorithm is implemented with general-purpose computation on graphics processing unit (GPGPU) for real-time processing owing to its suitability for parallel processing.

• Journal of Broadcast Engineering
• /
• v.21 no.2
• /
• pp.252-260
• /
• 2016

The performance of human detection system is affected by camera location and view angle. In 2D image acquired from such camera settings, humans are displayed in different sizes. Detecting all the humans with diverse sizes poses a difficulty in realizing a real-time system. However, if the size of a human in an image can be predicted, the processing time of human detection would be greatly reduced. In this paper, we propose a method that estimates human size by constructing an indoor scene in 3D space. Since the human has constant size everywhere in 3D space, it is possible to estimate accurate human size in 2D image by projecting 3D human into the image space. Experimental results validate that a human size can be predicted from the proposed method and that machine-learning based detection methods can yield the reduction of the processing time.

• Journal of Electrical Engineering and Technology
• /
• v.11 no.6
• /
• pp.1857-1862
• /
• 2016

Human activity recognition using depth information is an emerging and challenging technology in computer vision due to its considerable attention by many practical applications such as smart home/office system, personal health care and 3D video games. This paper presents a novel framework of 3D human body detection, tracking and recognition from depth video sequences using spatiotemporal features and modified HMM. To detect human silhouette, raw depth data is examined to extract human silhouette by considering spatial continuity and constraints of human motion information. While, frame differentiation is used to track human movements. Features extraction mechanism consists of spatial depth shape features and temporal joints features are used to improve classification performance. Both of these features are fused together to recognize different activities using the modified hidden Markov model (M-HMM). The proposed approach is evaluated on two challenging depth video datasets. Moreover, our system has significant abilities to handle subject's body parts rotation and body parts missing which provide major contributions in human activity recognition.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.8 no.5
• /
• pp.1690-1710
• /
• 2014

In this paper, we propose the rapid implementation of a 3-dimensional (3D) facial reconstruction from a single frontal face image and introduce a design for its application on a mobile device. The proposed system can effectively reconstruct human faces in 3D using an approach robust to lighting conditions, and a fast method based on a Canonical Correlation Analysis (CCA) algorithm to estimate the depth. The reconstruction system is built by first creating 3D facial mapping from a personal identity vector of a face image. This mapping is then applied to real-world images captured with a built-in camera on a mobile device to form the corresponding 3D depth information. Finally, the facial texture from the face image is extracted and added to the reconstruction results. Experiments with an Android phone show that the implementation of this system as an Android application performs well. The advantage of the proposed method is an easy 3D reconstruction of almost all facial images captured in the real world with a fast computation. This has been clearly demonstrated in the Android application, which requires only a short time to reconstruct the 3D depth map.

• Journal of the Korea Computer Graphics Society
• /
• v.18 no.2
• /
• pp.63-72
• /
• 2012

As BIM(Building Information Modeling) is widely used in the construction process the need for exploring building models realistically is also growing. In this paper, we present a 3D stereoscopic navigation method for virtual buildings considering virtual perception. We first find out factors that may cause virtual discomfort while navigating stereoscopic building models, and then develop a method for automatically adjusting the range of virtual camera separation. In addition, we measure each user's JND(Just Noticeable Difference) in depth to adjust virtual camera separation and movement. The presented method can be used for various architectural applications by creating user-customized 3D stereoscopic contents.

• Archives of design research
• /
• v.18 no.3 s.61
• /
• pp.209-218
• /
• 2005

Consciously or not, producers have to make many aesthetic choices in creative process of video production. If there are general acceptable aesthetic principles to make right choice it would be guideline of aesthetic decision to somewhat reduce mistakes and errors in the process. This paper proposes a theoretical approach on establishing the media aesthetic principle of 3D animation camera working, which is the most suitable for animation production context. We describe the Herbert Zettl's applied media aesthetics related directly to the camera, which is about the two-Dimensional field focusing on aspect radio and forces within the screen, three-dimensional field focusing on depth, volume, and four-dimensional field focusing on time and motion. In order to have theoretical approach we made an analysis on comparing a camera working of movie with 3D computer animation's one, and reconstructed these basic principles to be suited for the 3D animation production. When applied media aesthetics of the traditional camera working are applied to the 3D animation production, it could be an efficient guideline for it. Futhermore, if we develop the research for the relationship with various visual languages with the basis of these principles, the theory of creative picture composition method for the 3D animation production will be logically and systematically established.

• Journal of Korea Multimedia Society
• /
• v.20 no.2
• /
• pp.225-233
• /
• 2017

Line-laser beams are used for accurate measurement of 3D shape, which is robust to external illumination. For depth measurement, we project a line-laser beam across an object from the face and take an image of the beam on the object surface using a CCD camera at some angle with respect to the face. For shape measurement, we project parallel line-laser beams with narrow line to line distance. When a layer of thin materials attached to a cylinder is long narrow along its circumference, we can measure the shape of the layer with a small number of parallel line beams if we project line beams along the circumference of the cylinder. Measurement of the depth of the attached materials on a line-laser beam is based on the number of pixels between an imaginary line along the imaginary cylinder without the attached materials and the beam line along the materials attached to the cylinder. For this we need to localize the imaginary line in the captured image. In this paper, we model the shape of the line as an ellipse and localize the line with least square estimate. The proposed method results in smaller error (maximum 0.24mm) than a popular 3D depth camera (maximum 1mm).

• Journal of Broadcast Engineering
• /
• v.21 no.6
• /
• pp.957-964
• /
• 2016

Acquisition of reliable depth maps is a critical requirement in many applications such as 3D videos and free-viewpoint TV. Depth information can be obtained from the object directly using physical sensors, such as infrared ray (IR) sensors. Recently, Time-of-Flight (ToF) range camera including KINECT depth camera became popular alternatives for dense depth sensing. Although ToF cameras can capture depth information for object in real time, but are noisy and subject to low resolutions. Recently, filter-based depth up-sampling algorithms such as joint bilateral upsampling (JBU) and noise-aware filter for depth up-sampling (NAFDU) have been proposed to get high quality depth information. However, these methods often lead to texture copying in the upsampled depth map. To overcome this limitation, we formulate a convex optimization problem using higher order regularization for depth map upsampling. We decrease the texture copying problem of the upsampled depth map by using edge weighting term that chosen by the edge information. Experimental results have shown that our scheme produced more reliable depth maps compared with previous methods.