DOI QR코드

DOI QR Code

High Resolution Depth-map Estimation in Real-time using Efficient Multi-threading

효율적인 멀티 쓰레딩을 이용한 고해상도 깊이지도의 실시간 획득

  • Received : 2012.09.10
  • Accepted : 2012.11.19
  • Published : 2012.11.30

Abstract

A depth map can be obtained by projecting/capturing patterns of stripes using a projector-camera system and analyzing the geometric relationship between the projected patterns and the captured patterns. This is usually called structured light technique. In this paper, we propose a new multi-threading scheme for accelerating a conventional structured light technique. On CPUs and GPUs, multi-threading can be implemented by using OpenMP and CUDA, respectively. However, the problem is that their performance changes according to the computational conditions of partial processes of a structured light technique. In other words, OpenMP (using multiple CPUs) outperformed CUDA (using multiple GPUs) in partial processes such as pattern decoding and depth estimation. In contrast, CUDA outperformed OpenMP in partial processes such as rectification and pattern segmentation. Therefore, we carefully analyze the computational conditions where each outperforms the other and do use the better one in the related conditions. As a result, the proposed method can estimate a depth map in a speed of over 25 fps on $1280{\times}800$ images.

깊이지도를 구하는 방법 중 많이 사용되어지는 방법으로 줄무늬 패턴을 이용하는 방법이 존재한다. 이 방법은 프로젝터-카메라 시스템(Pro-Cam System)을 이용하며 프로젝터로 조사한 패턴을 카메라로 촬영하여 원래의 패턴과 촬영된 패턴간의 기하학적인 관계를 구하여 깊이지도를 구하는 방법이다. 본 논문에서는 이와 같이 구조광을 이용하는 깊이지도 획득 시스템에서 효과적으로 멀티 쓰레드를 사용하여 실시간 처리하는 것을 제안한다. 일반적으로 자주 사용되는 멀티 쓰레딩에는 CPU의 쓰레드를 이용하는 OpenMP와 GPU의 쓰레드를 이용하는 CUDA가 있다. 이 두 가지 기법은 수행하는데 차이점이 존재하기 때문에 상황에 따라 OpenMP가 더 좋은 효율을 보이는 부분이 있고 CUDA가 더 좋은 효율을 보이는 부분이 있다. 따라서 본 논문에서는 이 두 가지에 대해서 각 부분의 특성에 맞게 더 좋은 효율을 보이는 멀티 쓰레드를 적용하였다. 결과적으로 제안된 방법은 $1280{\times}800$의 영상에 대해 25fps 이상의 깊이지도를 획득할 수 있었다.

Keywords

References

  1. K. Sato and S. Inokuchi, "Three-dimensional surface measurement by space encoding range imaging," Journal of Robotic System, 2:27-39, 1985
  2. E. Horn and N. Kiryati, "Toward optimal structured light patterns," Image and Vision Computing, 17, 1999
  3. Daniel Scharstein and Richard Szeliski, "High-accuracy stereo depth map using structured light," IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, June, 2003
  4. S. Y. Chen, Y. F. Li, and J. W. Zhang "Vision processing for realtime 3-D data acquisition based on coded structured light", IEEE Trans. Image Process, vol.17, no. 2, pp. 167 - 176 , 2008 https://doi.org/10.1109/TIP.2007.914755
  5. Qiang Li, Moyuresh Biswas, Mark R. Pickering, and Michael R. Frater, "Dense depth estimation using adaptive structured light and cooperative algorithm," Computer Vision and Pattern Recognition Workshops, June, 2011
  6. Song Zhang and Peisen S. Huang, "High-resolution, real-time three-dimensional shape measurement," Optical Engineering, Vol.45, No.12, 2006
  7. S. Keerativittayanun, T. Kondo, P. Sira-uksorn, T. Phatrapornant, and M. Sato, "3D data acquisition using active stereo based on spatial neighbourhood technique," Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, May, 17-19, 2011
  8. L. Zhang, B. Curless, and S. M. Seitz, "Rapid shape acquisition using color structured light and multi-pass dynamic programming," 1st International Symposium on 3D Data processing, Visualization, and Transmission, Padova, Italy, June, 10-21, 2002
  9. Jordi Pages, joaquim Salvi, Christophe Collewet, and Joseq Forest, "Optimized De Bruijn pattern for one-shot shape acquisition," Image and Vision Computing Vol.23, No.8, pp. 707-720, August, 2005 https://doi.org/10.1016/j.imavis.2005.05.007
  10. N. H. Gabriel Falcao and J. Massich, "Plane-based calibration of a projector-camera system," technical report, VIBOT, 2008.
  11. C. Loop and Z. Zhang, "Computing Rectifying Homographies for Stereo Vision," IEEE Conference Computer Vision and Pattern Recognition, Fort Collins, CO, USA, pp. 125-131, June, 1999
  12. Adrian Ford and Alan Roberts, "Color Space Conversions," technical report, Westminster University, 1998
  13. H. Saito and T. Kanade, "Shape reconstruction in projective grid space from large number of images," IEEE Conference Computer Vision and Pattern recognition, vol. 2, pp. 49-54, 1999