• IEIE Transactions on Smart Processing and Computing
• /
• v.4 no.4
• /
• pp.224-230
• /
• 2015

This manuscript presents a real-time solution for 3D human body reconstruction with multiple RGB-D cameras. The proposed system uses four consumer RGB/Depth (RGB-D) cameras, each located at approximately $90^{\circ}$ from the next camera around a freely moving human body. A single mesh is constructed from the captured point clouds by iteratively removing the estimated overlapping regions from the boundary. A cell-based mesh construction algorithm is developed, recovering the 3D shape from various conditions, considering the direction of the camera and the mesh boundary. The proposed algorithm also allows problematic holes and/or occluded regions to be recovered from another view. Finally, calibrated RGB data is merged with the constructed mesh so it can be viewed from an arbitrary direction. The proposed algorithm is implemented with general-purpose computation on graphics processing unit (GPGPU) for real-time processing owing to its suitability for parallel processing.

• Journal of the Computational Structural Engineering Institute of Korea
• /
• v.31 no.4
• /
• pp.207-213
• /
• 2018

This paper investigates applicability of Microsoft $Kinect^{(R)}$, RGB-depth camera, to implement a 3D image and spatial information for sensing a target. The relationship between the image of the Kinect camera and the pixel coordinate system is formulated. The calibration of the camera provides the depth and RGB information of the target. The intrinsic parameters are calculated through a checker board experiment and focal length, principal point, and distortion coefficient are obtained. The extrinsic parameters regarding the relationship between the two Kinect cameras consist of rotational matrix and translational vector. The spatial images of 2D projection space are converted to a 3D images, resulting on spatial information on the basis of the depth and RGB information. The measurement is verified through comparison with the length and location of the 2D images of the target structure.

• Journal of Digital Convergence
• /
• v.14 no.11
• /
• pp.319-324
• /
• 2016

In the case of robot and Argument Reality applications using a camera in environments, a technology to estimate planes is a very important technology. A RGB-D camera can get a three-dimensional measurement data even in a flat which has no information of the texture of the plane;, however, there is an enormous amount of computation in order to process the point-cloud data of the image. Furthermore, it could not know the number of planes that are currently observed as an advance, also, there is an additional operation required to estimate a three dimensional plane. In this paper, we proposed the real-time method that decides the number of planes automatically and estimates the three dimensional plane by using the continuous data of an RGB-D camera. As experimental results, the proposed method showed an improvement of approximately 22 times faster speed compared to processing the entire data.

• The Journal of Korea Robotics Society
• /
• v.15 no.3
• /
• pp.293-300
• /
• 2020

The purpose of our sensor system is to transparentize the large hydraulic manipulators of a six-ton dual arm excavator from the operator camera view. Almost 40% of the camera view is blocked by the manipulators. In other words, the operator loses 40% of visual information which might be useful for many manipulator control scenarios such as clearing debris on a disaster site. The proposed method is based on a 3D reconstruction technology. By overlaying the camera image from front top of the cabin with the point cloud data from RGB-D (red, green, blue and depth) cameras placed at the outer side of each manipulator, the manipulator-free camera image can be obtained. Two additional algorithms are proposed to further enhance the productivity of dual arm excavators. First, a color correction algorithm is proposed to cope with the different color distribution of the RGB and RGB-D sensors used on the system. Also, the edge overlay algorithm is proposed. Although the manipulators often limit the operator's view, the visual feedback of the manipulator's configurations or states may be useful to the operator. Thus, the overlay algorithm is proposed to show the edge of the manipulators on the camera image. The experimental results show that the proposed transparentization algorithm helps the operator get information about the environment and objects around the excavator.

• Journal of Broadcast Engineering
• /
• v.25 no.7
• /
• pp.1095-1106
• /
• 2020

In this paper, we propose an algorithm that extracts a high-precision 3D skeleton using a model generated using a distributed RGB-D camera. When information about a 3D model is extracted through a distributed RGB-D camera, if the information of the 3D model is used, a skeleton with higher precision can be obtained. In this paper, in order to improve the precision of the 2D skeleton, we find the conditions to obtain the 2D skeleton well using the PCA. Through this, high-quality 2D skeletons are obtained, and high-precision 3D skeletons are extracted by combining the information of the 2D skeletons. Even though this process goes through, the generated skeleton may have errors, so we propose an algorithm that removes these errors by using the information of the 3D model. We were able to extract very high accuracy skeletons using the proposed method.

• Journal of Broadcast Engineering
• /
• v.26 no.3
• /
• pp.247-257
• /
• 2021

This paper proposes a new technique for calibrating a multi-view RGB-D camera using a 3D (dimensional) skeleton. In order to calibrate a multi-view camera, consistent feature points are required. In addition, it is necessary to acquire accurate feature points in order to obtain a high-accuracy calibration result. We use the human skeleton as a feature point to calibrate a multi-view camera. The human skeleton can be easily obtained using state-of-the-art pose estimation algorithms. We propose an RGB-D-based calibration algorithm that uses the joint coordinates of the 3D skeleton obtained through the posture estimation algorithm as a feature point. Since the human body information captured by the multi-view camera may be incomplete, the skeleton predicted based on the image information acquired through it may be incomplete. After efficiently integrating a large number of incomplete skeletons into one skeleton, multi-view cameras can be calibrated by using the integrated skeleton to obtain a camera transformation matrix. In order to increase the accuracy of the calibration, multiple skeletons are used for optimization through temporal iterations. We demonstrate through experiments that a multi-view camera can be calibrated using a large number of incomplete skeletons.

• Advances in robotics research
• /
• v.2 no.2
• /
• pp.129-140
• /
• 2018

With the increasing demand for the development of human pose estimation, such as human-computer interaction and human activity recognition, there have been numerous approaches to detect the 2D poses of people in images more efficiently. Despite many years of human pose estimation research, the estimation of human poses with images remains difficult to produce satisfactory results. In this study, we propose a robust 2D human body pose estimation method using an RGB camera sensor. Our pose estimation method is efficient and cost-effective since the use of RGB camera sensor is economically beneficial compared to more commonly used high-priced sensors. For the estimation of upper-body joint positions, semantic segmentation with a fully convolutional network was exploited. From acquired RGB images, joint heatmaps accurately estimate the coordinates of the location of each joint. The network architecture was designed to learn and detect the locations of joints via the sequential prediction processing method. Our proposed method was tested and validated for efficient estimation of the human upper-body pose. The obtained results reveal the potential of a simple RGB camera sensor for human pose estimation applications.

• International conference on construction engineering and project management
• /
• 2022.06a
• /
• pp.328-335
• /
• 2022

With the advance of robot capabilities and functionalities, construction robots assisting construction workers have been increasingly deployed on construction sites to improve safety, efficiency and productivity. For close-proximity human-robot collaboration in construction sites, robots need to be aware of the context, especially construction worker's behavior, in real-time to avoid collision with workers. To recognize human behavior, most previous studies obtained 3D human poses using a single camera or an RGB-depth (RGB-D) camera. However, single-camera detection has limitations such as occlusions, detection failure, and sensor malfunction, and an RGB-D camera may suffer from interference from lighting conditions and surface material. To address these issues, this study proposes a novel method of 3D human pose estimation by extracting 2D location of each joint from multiple images captured at the same time from different viewpoints, fusing each joint's 2D locations, and estimating the 3D joint location. For higher accuracy, the probabilistic representation is used to extract the 2D location of the joints, considering each joint location extracted from images as a noisy partial observation. Then, this study estimates the 3D human pose by fusing the probabilistic 2D joint locations to maximize the likelihood. The proposed method was evaluated in both simulation and laboratory settings, and the results demonstrated the accuracy of estimation and the feasibility in practice. This study contributes to ensuring human safety in close-proximity human-robot collaboration by providing a novel method of 3D human pose estimation.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.43 no.3 s.309
• /
• pp.25-33
• /
• 2006

This paper proposed device-independent color matching algorithm based on the 3D RGB lookup table (LUT) between mobile camera and mobile LCD (Liquid Crystal Display) to improve the color-fidelity. Proposed algorithm is composed of thee steps, which is device characterization, gamut mapping, 3D RGB-LUT design. First, the characterization of mobile LCD is executed using the sigmoidal function, different from conventional method such as GOG (Gain Offset Gamma) and S-curve modeling, based on the observation of electro-optical transfer function of mobile LCD. Next, mobile camera characterization is conducted by fitting the digital value of GretagColor chart captured under the daylight environment (D65) and tristimulus values (CIELAB) using the polynomial regression. However, the CIELAB values estimated by polynomial regression exceed the maximum boundary of the CIELAB color space. Therefore, these values are corrected by linear compression of the lightness and chroma. Finally, gamut mapping is used to overcome the gamut difference between mobile camera and moible LCD. To implement the real-time processing, 3D RGB-LUT is designed based on the 3D RGB-LUT and its performance is evaluated and compared with conventional method.

• Journal of the Korea institute for structural maintenance and inspection
• /
• v.26 no.6
• /
• pp.157-166
• /
• 2022

Structural supervision on the construction site has been performed based on visual inspection, which is highly labor-intensive and subjective. In this study, the remote technique was developed to improve the efficiency of the measurements on rebar spacing using a 360° camera and reconstructed 3D models. The proposed method was verified by measuring the spacings in reinforced concrete structure, where the twelve locations in the construction site (265 m²) were scanned within 20 seconds per location and a total of 15 minutes was taken. SLAM, consisting of SIFT, RANSAC, and General framework graph optimization algorithms, produces RGB-based 3D and 3D point cloud models, respectively. The minimum resolution of the 3D point cloud was 0.1mm while that of the RGB-based 3D model was 10 mm. Based on the results from both 3D models, the measurement error was from 10.8% to 0.3% in the 3D point cloud and from 28.4% to 3.1% in the RGB-based 3D model. The results demonstrate that the proposed method has great potential for remote structural supervision with respect to its accuracy and objectivity.