• Journal of the HCI Society of Korea
• /
• v.2 no.2
• /
• pp.1-9
• /
• 2007

In this paper, we propose a reconstruction technique of the lost hair region for 3D human actor modeling. An active depth sensor system can simultaneously capture both color and geometry information of any objects in real-time. However, it cannot acquire some regions whose surfaces are shiny and dark. Therefore, to get a natural 3D human model, the lost region in depth image should be recovered, especially human hair region. The recovery is performed using both color and depth images. We find out the hair region using color image first. After the boundary of hair region is estimated, the inside of hair region is estimated using an interpolation technique and closing operation. A 3D mesh model is generated after performing a series of operations including adaptive sampling, triangulation, mesh smoothing, and texture mapping. The proposed method can generate recovered 3D mesh stream automatically. The final 3D human model allows the user view interaction or haptic interaction in realistic broadcasting system.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.21 no.3
• /
• pp.59-66
• /
• 2021

Humans mainly recognize surrounding objects using visual and auditory information among the five senses (sight, hearing, smell, touch, taste). Major research related to the latest object recognition mainly focuses on analysis using image sensor information. In this paper, after emitting various chirp audio signals into the observation space, collecting echoes through a 2-channel receiving sensor, converting them into spectral images, an object recognition experiment in 3D space was conducted using an image learning algorithm based on deep learning. Through this experiment, the experiment was conducted in a situation where there is noise and echo generated in a general indoor environment, not in the ideal condition of an anechoic room, and the object recognition through echo was able to estimate the position of the object with 83% accuracy. In addition, it was possible to obtain visual information through sound through learning of 3D sound by mapping the inference result to the observation space and the 3D sound spatial signal and outputting it as sound. This means that the use of various echo information along with image information is required for object recognition research, and it is thought that this technology can be used for augmented reality through 3D sound.