International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Fast key-frame extraction for 3D reconstruction from a handheld video

  • Choi, Jongho (Department of Electronic Engineering, Kwangwoon University) ;
  • Kwon, Soonchul (Graduate School of Information and Contents, Kwangwoon University) ;
  • Son, Kwangchul (Graduate School of Information and Contents, Kwangwoon University) ;
  • Yoo, Jisang (Department of Electronic Engineering, Kwangwoon University)
  • Received : 2016.09.10
  • Accepted : 2016.10.07
  • Published : 2016.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In order to reconstruct a 3D model in video sequences, to select key frames that are easy to estimate a geometric model is essential. This paper proposes a method to easily extract informative frames from a handheld video. The method combines selection criteria based on appropriate-baseline determination between frames, frame jumping for fast searching in the video, geometric robust information criterion (GRIC) scores for the frame-to-frame homography and fundamental matrix, and blurry-frame removal. Through experiments with videos taken in indoor space, the proposed method shows creating a more robust 3D point cloud than existing methods, even in the presence of motion blur and degenerate motions.

Keywords

References

  1. L. Ling, Ian S. Burrent, E. Cheng, "A dense 3D reconstruction approach from uncalibrated video sequences" ICMEW, pp. 587-592, 2012.
  2. J. Frahm, M. Pollefeys, S. Lazebnik, D. Gallup, B. Clipp, R. Raguram, C. Wu, C. Zach, T. Johnson, "Fast robust large-scale mapping from video and internet photo collections" ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 65, No. 6, pp. 538-549, 2010. https://doi.org/10.1016/j.isprsjprs.2010.08.009
  3. N. Snavely, S.M. Seitz, R. Szeliski, "Modeling the world from Internet photo collections", International Journal of Computer Vision, Vol. 80, No. 2, pp. 189-210, 2008. https://doi.org/10.1007/s11263-007-0107-3
  4. M. Pollefeys et al., "Detailed real-time urban 3D reconstruction from video", International Journal of Computer Vision, Vol. 78, pp. 143-167, 2008. https://doi.org/10.1007/s11263-007-0086-4
  5. S. Gibson, J. Cook, T. Howard, R. Hubbold, D. Oram. "Accurate camera calibration for off-line, video-based augmented reality", in: IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR), Darmstadt, Germany, 2002.
  6. J.K. Seo, S.H. Kim, C.W. Jho, H.K. Hong, "3D Estimation and Key-Frame Selection for Match Move", ITC-CSCC: International Technical Conference on Circuits Systems, Computers and Communications, pp. 1282-1285, July 2003.
  7. Y.H. Seo, S.H. Kim, K.S. Doo, J.S. Choi, "Optimal keyframe selection algorithm for three-dimensional reconstruction in uncalibrated multiple images", Journal of the Society of Photo-Optical Instrumentation Engineers, Vol. 47, No. 5, pp. 53201-53400, 2008
  8. M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, R. Koch, "Visual modeling with a hand-held camera", International Journal of Computer Vision, Vol. 59, No. 3, pp. 207-232. https://doi.org/10.1023/B:VISI.0000025798.50602.3a
  9. P.H.S. Torr, A.W. Fitzgibbon, A. Zisserman, "Maintaining multiple motion model hypotheses over many views to recover matching and structure", in: Proc. The 6th International Conference on Computer Vision, Bombay, India, pp. 485-491, 1998.
  10. M.T. Ahmed, M.N. Dailey, J.L. Landabaso, N. Herrero, "Robust key frame extraction for 3D reconstruction from video streams", in: Proc. The VISAPP, pp. 231-236, 2010.
  11. R. Fergus, B. Singh, A. Hertzmann, S.T. Roweis, W.T. Freeman, "Removing camera shake from a single photograph", ACM Transactions on Graphics, Vol. 25, No. 3, pp. 787-794, 2006. https://doi.org/10.1145/1141911.1141956
  12. Y. Matsushita, E. Ofek, X. Tang, H.Y. Shum, "Full-frame video stabilization", in: Proc. Computer Vision and Pattern Recognition, pp. 50-57, 2005.
  13. S. Cho, J. Wang, S. Lee, "Video deblurring for hand-held cameras using patch-based synthesis", ACM Transactions on Graphics, Vol. 31, No. 4, pp. 1-9, 2012.
  14. S. Yang, M. Lizhuang, "Detecting and Removing the Motion Blurring from Video Clips", I.J.Modern Education and Computer Science, Vol. 1, pp. 17-23, January 2010.
  15. Changchang Wu, "Towards Linear-time Incremental Structure from Motion", in: Proc. International Conference on 3D Vision, pp. 127-134, 2013.
  16. Changchang Wu, "VisualSFM: A Visual Structure from Motion System", http://ccwu.me/vsfm/, 2011