IEIE Transactions on Smart Processing and Computing

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Real-time Full-view 3D Human Reconstruction using Multiple RGB-D Cameras

  • Yoon, Bumsik (Department of Visual Display, Samsung Electronics) ;
  • Choi, Kunwoo (Department of Electronics and Computer Engineering, Hanyang University) ;
  • Ra, Moonsu (Department of Electronics and Computer Engineering, Hanyang University) ;
  • Kim, Whoi-Yul (Department of Electronics and Computer Engineering, Hanyang University)
  • Received : 2015.07.15
  • Accepted : 2015.08.24
  • Published : 2015.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

This manuscript presents a real-time solution for 3D human body reconstruction with multiple RGB-D cameras. The proposed system uses four consumer RGB/Depth (RGB-D) cameras, each located at approximately $90^{\circ}$ from the next camera around a freely moving human body. A single mesh is constructed from the captured point clouds by iteratively removing the estimated overlapping regions from the boundary. A cell-based mesh construction algorithm is developed, recovering the 3D shape from various conditions, considering the direction of the camera and the mesh boundary. The proposed algorithm also allows problematic holes and/or occluded regions to be recovered from another view. Finally, calibrated RGB data is merged with the constructed mesh so it can be viewed from an arbitrary direction. The proposed algorithm is implemented with general-purpose computation on graphics processing unit (GPGPU) for real-time processing owing to its suitability for parallel processing.

Keywords

References

  1. M. Botsch, et al., Polygon Mesh Processing, AK Peters, London, 2010.
  2. H. Hoppe, et al., "Surface reconstruction from unorganized points," SIGGRAPH '92, 1992.
  3. R. Mencl and H. Muller. Graph-based surface reconstruction using structures in scattered point sets. In Proceedings of CGI '98 (Computer Graphics International), pp. 298-311, June, 1998.
  4. B. Curless and M. Levoy, "A volumetric method for building complex models from range images," SIGGRAPH '96, 1996.
  5. M. Kazhdan, M. Bolitho, and H. Hoppe, "Poisson Surface Reconstruction," Proc. Symp. Geometry Processing, 2006.
  6. Microsoft Kinect
  7. D. Alexiadis, D. Zarpalas, and P. Daras, "Real-time, full 3-D reconstruction of moving foreground objects from mul-tiple consumer depth cameras," IEEE Trans on Multimedia, vol. 15, pp. 339-358, Feb. 2013. https://doi.org/10.1109/TMM.2012.2229264
  8. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," in Proc. of the ICCV, 1998.
  9. J. Smisek, M. Jancosek, and T. Pajdla, "3D with Kinect," 2011 ICCV Workshops, pp. 1154-1160, Nov. 2011.
  10. P. Besl and N. McKay, "A Method for Registration of 3-D Shapes," IEEE Trans. Patten Analysis and Machine Intelligence, vol. 14, pp. 239-256, 1992. https://doi.org/10.1109/34.121791
  11. R. Toldo, A. Beinat, and F. Crosilla, "Global registration of multiple point clouds embedding the generalized procrustes analysis into an ICP framework," in 3DPVT 2010 Conf., Paris, May 17- 20, 2010.
  12. S. Bouaziz, A. Tagliasacchi, and M. Pauly, "Sparse Iterative Closest Point," Computer Graphics Forum, vol. 32, no. 5, pp. 1-11, 2013.
  13. K. Shoemake, "Animating rotation with quaternion curves," in Proc. of the SIGGRAPH '85, 1985, pp. 245-254.
  14. NVidia CUDA
  15. CGAL
  16. Sparse ICP

Cited by

  1. Depth map generation using a single image sensor with phase masks vol.24, pp.12, 2016, https://doi.org/10.1364/OE.24.012868
  2. Optical range-finding system using a single-image sensor with liquid crystal display aperture vol.41, pp.22, 2016, https://doi.org/10.1364/OL.41.005154
  3. Robust Video Stabilization Using Particle Keypoint Update and l1-Optimized Camera Path vol.17, pp.2, 2017, https://doi.org/10.3390/s17020337