Journal of the Institute of Electronics Engineers of Korea SP (대한전자공학회논문지SP)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Projective Reconstruction Method for 3D modeling from Un-calibrated Image Sequence

비교정 영상 시퀀스로부터 3차원 모델링을 위한 프로젝티브 재구성 방법

  • Hong Hyun-Ki (Dept. of Image Eng., Graduate School of Advanced Imaging Science Multimedia & Film, chung-Ang Univ.) ;
  • Jung Yoon-Yong (Dept. of Image Eng., Graduate School of Advanced Imaging Science Multimedia & Film, chung-Ang Univ.) ;
  • Hwang Yong-Ho (Dept. of Image Eng., Graduate School of Advanced Imaging Science Multimedia & Film, chung-Ang Univ.)
  • 홍현기 (중앙대학교 첨단영상대학원 영상공학과) ;
  • 정윤용 (중앙대학교 첨단영상대학원 영상공학과) ;
  • 황용호 (중앙대학교 첨단영상대학원 영상공학과)
  • Published : 2005.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

3D reconstruction of a scene structure from un-calibrated image sequences has been long one of the central problems in computer vision. For 3D reconstruction in Euclidean space, projective reconstruction, which is classified into the merging method and the factorization, is needed as a preceding step. By calculating all camera projection matrices and structures at the same time, the factorization method suffers less from dia and error accumulation than the merging. However, the factorization is hard to analyze precisely long sequences because it is based on the assumption that all correspondences must remain in all views from the first frame to the last. This paper presents a new projective reconstruction method for recovery of 3D structure over long sequences. We break a full sequence into sub-sequences based on a quantitative measure considering the number of matching points between frames, the homography error, and the distribution of matching points on the frame. All of the projective reconstructions of sub-sequences are registered into the same coordinate frame for a complete description of the scene. no experimental results showed that the proposed method can recover more precise 3D structure than the merging method.

비교정 영상 시퀀스(un-calibrated sequence)로부터 대상 장면을 재구성하는 연구는 컴퓨터 비젼에서 중요한 주제이다. 3차인 정보론 유클리드 공간에서 재구성하기 위해 프로젝티브(projective) 재구성이 선행되며, 이는 병합(merging)방법과 분해 (factorization)방법으로 나뉜다. 분해방법은 카메라 투영행렬과 3차원 구조정보를 한 번에 계산하기 때문에 계산속도가 빠르며, 병합방법의 단점인 오차의 누적 문제를 해결할 수 있다. 그러나 사용되는 일치점(correspondence)이 모든 영상 시퀀스에 존재한다는 가정으로 인해 긴 시퀀스에 적용하기 어렵다. 본 논문에서는 영상 시퀀스를 몇 개의 그룹으로 나누고 각 그룹을 분해 법으로 프로젝티브 재구성을 한 다음, 하나의 프로젝티브 공간으로 다시 구성하는 새로운 방법을 제안하였다. 시퀀스에서 그룹을 결정하기 위해 매칭점의 개수, 평면사영변환(homography) 오차, 영상 내 매칭점의 분포를 함께 고려했으며, 병합방법에 비해 카메라 파라미터의 오차 누적이 적고 계산속도면에서도 우수함을 실험을 통해 확인하였다.

Keywords

References

  1. A. W. Fitzgibbon and A. Zisserman, 'Automatic camera recovery for closed or open image sequences,' In proc of 6th European Conference on Computer Vision, pp. 311-326, June 1998 https://doi.org/10.1007/BFb0055675
  2. K. Cornelis, M. Pollefeys, M. Vergauwen, and L. V. Gool, 'Augmented reality from uncalibrated video sequences,' In proc SMILE 2000, Lecture Notes in Computer Science, vol. 2018, pp. 144-160, 2001
  3. S. Gibson, J. Cook, T. Howard, R. Hubbold, and D. Oram, 'Accurate camera calibration for off-line, video-based augmented reality,' In proc. IEEE and ACM ISMAR, Darmstadt, Germany, Sep.2002 https://doi.org/10.1109/ISMAR.2002.1115068
  4. A. Chiuso, P. Favaro, H. Jin, and S. Soatto, 'Motion and structure causally integrated over time,' IEEE Trans. on Pattern Matching and Machine Intelligence, vol. 24, no .4, pp. 523-535, April 2002 https://doi.org/10.1109/34.993559
  5. P. Sturm and B. Triggs, 'A factorization based algorithm for multi-image projective structure and motion,' In proc of 4th European Conference on Computer Vision Cambridge, England, pp. 707-720, April 1996
  6. R. Hartley and A. Zisserman, Mutiple View Geometry in Computer Vision, Cambrige Univ. Press. 2000
  7. Takeo Kanade, Masatoshi Okutomi, 'A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,' IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 16, no. 9, Sept, 1994 https://doi.org/10.1109/34.310690
  8. S. Gibson, R. J. Hubbold, J. Cook, and T. Howard, 'Interactive reconstruction of virtual environments from video sequences,' Computer Graphics, vol. 27, pp. 293-301, 2003 https://doi.org/10.1016/S0097-8493(02)00285-6
  9. Debevec, P., C. Taylor and J. Malik, 'Modeling and Rendering Architecture from Photographs,' SIGGRAPH96, pp. 11-20, 1996 https://doi.org/10.1145/237170.237191
  10. T. Sato, M. Kanbara, N. Yokoya, and H. Takemura, 'Dense 3-D reconstruction of outdoor scene by hundreds-baseline stereo using a hand-held video camera,' In proc. of IEEE Workshop on Stereo and Multi-baseline vision, 2001 https://doi.org/10.1109/SMBV.2001.988763
  11. A. Heyden, R. Berthilsson, and G. Sparr, 'An iterative factorization method for projective structure and motion from image sequences,' Image and Vision Computing, pp. 981-991, 1999 https://doi.org/10.1016/S0262-8856(99)00002-5
  12. B. Triggs, 'Autocalibration and the absolute quadric,' In proc of IEEE CVPR, pp. 609-614, 1997 https://doi.org/10.1109/CVPR.1997.609388
  13. R. Hartley, 'In Defence of the 8-Point algorithm,' In proc. IEEE ICCV, pp. 1064-1070, 1995 https://doi.org/10.1109/ICCV.1995.466816
  14. Z. Zhang, 'Computing rectifying homographies for stereo vision,' In proc. of IEEE CVPR, pp. 125-131, Fort Collins Colorado, June 1999 https://doi.org/10.1109/CVPR.1999.786928
  15. C. J. Tasi and A. K. Katsaggelos, ' Dense Disparity Estimation with a Divide-and-Conquer Disparity Space Image Technique,' IEEE, Trans. Multimedia, Vol.1, No.1, pp.18-29, March, 1999 https://doi.org/10.1109/6046.748168
  16. S. S. Intille and A. F. Bobick, 'Disparity-space images and large occlusion stereo,' 2nd European Conference on Computer Vision, pp. 179-186, Stockholm Sweden, 1994