Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Structure and Motion Estimation with Expectation Maximization and Extended Kalman Smoother for Continuous Image Sequences

부드러운 카메라 움직임을 위한 EM 알고리듬을 이용한 삼차원 보정

  • Published : 2004.02.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper deals with the problem of estimating structure and motion from long continuous image sequences, applying the Expectation Maximization algorithm based on extended Kalman smoother to impose the time-continuity of the motion parameters. By repeatedly estimating the state transition matrix of the dynamic equation and the parameters of noise processes in the dynamic and measurement equations, this optimization gives the maximum likelihood estimates of the motion and structure parameters. Practically, this research is essential for dealing with a long video-rate image sequence with partially unknown system equation and noise. The algorithm is implemented and tested for a real image sequence.

이 논문은 카메라가 연속적으로 움직일 때 그 카메라로부터 얻은 동영상을 분석하여 카메라의 움직임에 대한 정보와 영상내의 구조물의 삼차원 정보를 계산하는 알고리듬에 대한 것이다. 일반적으로 불 연속한 위치에서 얻은 영상의 집합으로부터 삼차원정보 및 카메라 정보를 얻는 경우에는 카메라의 움직임에 대한 제약조건이 필요 없지만, 비디오 카메라를 이용하여 동영상을 취득하는 경우에는 항상 카메라의 움직임이 부드러워야 한다는 조건이 따라 붙는다. 따라서, 이 논문에서는 ‘부드러운 움직임을 가지는 카메라’라는 제약조건을 포함하는 카메라 및 삼차원정보의 최적화 과정에 대하여 연구하였다. 목적하는 바를 얻기 위하여 Expectation-Maximization 방법을 사용하여 카메라의 움직임에 대한 모델 파라메터를 동시에 추정하였는데, 이를 위하여 Extended Kalman Filter 와 Extended Kalman Smoother를 적용하였다. 이 연구는 길이가 긴 비디오 영상열의 비젼 해석에 기본이 된다. 실제 영상을 이용하여 실험한 결과를 보였다.

Keywords

References

  1. D. Nister. Reconstruction from uncalibrated sequences with a hierachy of trifocal tensors. In Proc. European Conf. on Computer Vision, 2000
  2. C.Matsunaga and K.Kanatani. Calibration of a moving camera using a planer patter: Optimal computation, reliability evaluation and stabilization by model selection. In Proc. European Conf. on Computer Vision, 2000
  3. G. Simon and M.-O. Berger. Registeration with a moving zoom lens camera for augmented reality applications. In Proc. European Conf. on Computer Vision, 2000 https://doi.org/10.1007/3-540-45053-X_37
  4. Kurt Cornelis, Marc Pollefeys, and Maarten Vergauwen adn Luc Van Gool. Augmented reality using uncalibrated video sequences. Second Workshop on Structure from Multiple Images of Large Scale Environments , In conjunction with ECCV 2000, Dublin, Ireland, 2000
  5. G. Simon, A.W. Fitzgibbon, and A. Zisserman. Markerless tracking using planar structures in the scene. In IEEE and ACM International Symposium on Augmented Reality. IEEE Computer Society Press, 2000 https://doi.org/10.1109/ISAR.2000.880935
  6. V. Lepetit and M.-O. Berger. Handling occlusion in augmented reality systems: A semi-automatic method. IEEE and ACM International Symposium on Augmented Reality. IEEE Computer Society Press, 2000 https://doi.org/10.1109/ISAR.2000.880937
  7. Y. Seo and K. S. Hong. Weakly calibrated video-based augmented reality:embedding and rendering through virtual camera. In IEEE and ACM International Symposium on Augmented Reality. IEEE Computer Society Press, 2000 https://doi.org/10.1109/ISAR.2000.880936
  8. Philip F. McLauchlan. A batch/reccursive algorithm for 3D scene reconstruction. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000 https://doi.org/10.1109/CVPR.2000.854948
  9. B. Triggs, P.F. Mc L auchlan, R.I .Hartley, and A.W. Fitzgibbon. Bundle adjustment - a modern synthesis. In Vision Algorithms: Theory and Practice, Lecture Notes in Computer Science1883. Springer-Verlag, 1999
  10. Ezio Malis and Roberto Cipolla. Multi-view constraints between collineations: application to self-calibation from unknown planar structures. Proc. 6th European Conf. on Computer Vision, Dublin, Ireland. Springer-Verlag, 2000
  11. Paulo Mendonca and Roberto Cipolla. A simple technique for self-calibration. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999 https://doi.org/10.1109/CVPR.1999.786984
  12. B. Triggs. Autocalibration from planar scenes. In Proc. of European Conf. on Computer Vision, 1998
  13. M. Pollefeys, R. Koch, L. Van Gool. Selfcalibration and metric reconstruction in spite of varying and unknown internal camera parameters. In Proc. 6th Int. Conf. on Computer Vision, Mumbai, India, 1998 https://doi.org/10.1109/ICCV.1998.710705
  14. B. Triggs. Autocalibration and the absolute quadric. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997 https://doi.org/10.1109/CVPR.1997.609388
  15. A. Heyden and K. A. Strom. Euclidean reconstruction from constant intrinsic parameters. In ICPR'96, pages 339-343. IEEE Computer Society Press, 1996 https://doi.org/10.1109/ICPR.1996.546045
  16. M. Pollefeys, L. Van Gool, and M. Proesmans. Euclidean 3d reconstruction from image sequences with variable focal lengths. In Proc. 4th European Conf. on Computer Vision, Cambridge, UK, pages 31-42, 1996 https://doi.org/10.1007/BFb0015521
  17. P. F. McLauchlan and D. W. Murray. A unifying framework for structure and motion recovery from image sequences. In Proc. 5th Int. Conf. on Computer Vision, MIT, Boston, MA, pages 314-320. IEEE Computer Society Press, Los Alamitos, California, 1995 https://doi.org/10.1109/ICCV.1995.466923
  18. A. Azarbayejani and A.P. Pentland. Recursive estimation of motion, struture and focal length. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(6):562-575, 1995 https://doi.org/10.1109/34.387503
  19. Paul Beardsley, Andrew Zisserman, and David Murray. Sequential updating of projective and affine structure from motion, Technical Report OUEL 2070/95, Dept. of Engineering Science, University of Oxford, 1995
  20. N. Cui, J. Weng, and P. Cohen. Recursive-batch estimation of motion and structure from monocular image sequences. Computer Vision, Graphics, and Image Processing, 59(2):154-170, 1993 https://doi.org/10.1006/ciun.1994.1010
  21. A. Gelb, editor. Applied Optimal Estimation. MIT Press, 1974
  22. P.S. Maybeck, editor. Stochastic models, estimation, and control VOLUME 2. Academic Press, 1979
  23. H.E. Rauch, E. Tung, and C.T. Striebel. Maximum likelihood estimates of linear dynamic systems. AIAA Journal, 3(8):1445-1450, 1965 https://doi.org/10.2514/3.3166
  24. A.P. Dempster, N.M. Laird, and D.B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society Series B, 39:1-38, 1977
  25. Todd K. Moon. The expectation-maximization algorithm. IEEE Signal Processing Magazine, 13(6):47-60, 1996 https://doi.org/10.1109/79.543975
  26. Z. Ghahramani and G.E. Hinton. Parameter estimation for linear dynamical systems. Technical Report CRG-TR-96-2, University of Toronto, 1996
  27. M.N. Nando de Freitas and Andrew Gee. Dynamic learning with the EM algorithm for neural networks. VLSI Signal Processing Systems, pages 119-131, 2000 https://doi.org/10.1023/A:1008103718973
  28. Z. Ghahramani and M.J. Beal. Learning nonlinear dynamical systems using an em algorithm. In M.S. Kearns, S.A. Solla, and D.A. Cohn, editors, Advances in Neural Information Processing Systems, V.11. MIT Press, 1999
  29. Y. Seo and K.S. Hong. Structure and motion estimation with expectation maximization and extended Kalman smoother for continuous image sequence, In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, Dec. 2001 https://doi.org/10.1109/CVPR.2001.990660
  30. K. Kanatani, Statistical Optimization for Geometric Computation: Theory and Practice. Elsevier Science, 1996
  31. Anders Heyden, Rikard Berthilsson, and Gunnar Sparr. An iterative factorization method for projective structure and motion from image sequences. Image and Vision Computing, 17(5), 1999 https://doi.org/10.1016/S0262-8856(99)00002-5
  32. Y. Seo, Anders Heyden, and R. Cipolla. A linear iterative method for auto-calibration using the DAC equation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, Dec. 2001 https://doi.org/10.1109/CVPR.2001.990577
  33. O. Faugeras. Three-Dimensional Computer Vision. MIT Press, Cambridge, Mass, 1993
  34. A.J. Davison and N. Kita. Sequential localisation and map-building in computer vision and robotics. Second Workshop on Structure from Multiple Images of Large Scale Environments, In conjunction with ECCV 2000, Dublin, Ireland, 2000