Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Video Deblurring using Camera Motion Estimation and Patch-wise Deconvolution

카메라 움직임 추정 및 패치 기반 디컨볼루션을 이용한 동영상의 번짐 현상 제거 방법

  • Jeong, Woojin (Dept. of Computer Science & Engineering, Hanyang University) ;
  • Park, Jin Wook (Dept. of Computer Science & Engineering, Hanyang University) ;
  • Lee, Jong Min (Dept. of Computer Science & Engineering, Hanyang University) ;
  • Song, Tae Eun (Optronics System Group, Samsung Thales) ;
  • Choi, Wonju (Optronics System Group, Samsung Thales) ;
  • Moon, Young Shik (Dept. of Computer Science & Engineering, Hanyang University)
  • 정우진 (한양대학교 컴퓨터공학과) ;
  • 박진욱 (한양대학교 컴퓨터공학과) ;
  • 이종민 (한양대학교 컴퓨터공학과) ;
  • 송태언 (삼성탈레스 전자광학체계그룹) ;
  • 최원주 (삼성탈레스 전자광학체계그룹) ;
  • 문영식 (한양대학교 컴퓨터공학과)
  • Received : 2014.08.12
  • Accepted : 2014.11.30
  • Published : 2014.12.25
Download PDF
⟨ Previous Next ⟩

Abstract

Undesired camera shaking can make a blur effect, which causes a degradation of video quality. We propose an efficient method of removing the blur effects on video captured from a single camera. The proposed method has a sequential process that is applied to each frame. The first stage is to estimate the camera motion for each frame. In order to estimate the camera motion, we compute the optical flow using 3 consecutive frames. Then a patch-wise image deconvolution is applied. During the deconvolution, edge prediction is used to improve the quality of image deconvolution. After patch-wise image deconvolution, deblurred patches are integrated into an image to produce a deblurred frame. The above process is performed for each frame. The experimental result shows that the proposed method removes the blur effect efficiently.

동영상 촬영 시 급격한 카메라의 흔들림은 의도하지 않은 번짐 현상을 발생시켜 동영상의 품질을 낮추는 원인이 된다. 따라서 본 논문에서는 동영상의 품질을 높이기 위해 동영상에서 카메라 흔들림으로 인해 발생한 번짐 현상을 제거하는 방법을 제안한다. 제안하는 방법은 매 프레임 별로 이루어진다. 각 프레임마다 이전 프레임과 현재 프레임, 다음 프레임을 이용하여 카메라 움직임을 계산한다. 그리고 카메라의 움직임을 바탕으로 점 확산 함수를 계산하고 프레임을 패치 단위로 쪼개어 패치별 번짐 현상을 제거한다. 이때 품질을 높이기 위하여 번짐 영상으로부터 외곽선을 예측하는 방법을 사용한다. 번짐 현상이 제거된 패치는 다시 하나의 프레임으로 합한다. 실험 결과를 통해 제안하는 방법이 동영상에서의 카메라 흔들림으로 인한 번짐 현상을 효과적으로 제거함을 확인하였다.

Keywords

References

  1. R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, "Removing camera shake from a single photograph," ACM Transaction on Graphics(TOG), Vol. 25, No. 3, pp. 787-794, 2006
  2. Q. Shan, J. Jia, and A. Agarwala, "High-quality motion deblurring from a single image," ACM Transactions on Graphics (TOG), Vol. 27, No. 3, 2008.
  3. S. H. Cho, and S. Y. Lee, "Fast motion deblurring," ACM Transactions on Graphics (TOG), Vol. 28, No. 5, 2009.
  4. L. Xu, and J. Jia, "Two-phase kernel estimation for robust motion deblurring," In Computer Vision-ECCV 2010, pp. 157-170, Springer Berlin Heidelberg, 2010.
  5. L. Yuan, J. Sun, L. Quan, and H. Y. Shum, "Image deblurring with blurred/noisy image pairs," ACM Transactions on Graphics (TOG), Vol. 26, No. 3, 2007.
  6. N. Joshi, S. B. Kang, C. L. Zitnick, and R. Szeliski, "Image deblurring using inertial measurement sensors," ACM Transactions on Graphics (TOG), Vol. 29, No. 4, 2010.
  7. M. Ben-Ezra, and S. K. Nayar, "Motion deblurring using hybrid imaging," Computer Vision and Pattern Recognition (CVPR), Vol. 1, 2003.
  8. Y. W. Tai, H. Du, M. S. Brown, and S. Lin, "Image/video deblurring using a hybrid camera," Computer Vision and Pattern Recognition (CVPR), Vol. 1, 2003.
  9. Y. Li, S. B. Kang, N. Joshi, S. and M. Seitz, "Generating sharp panoramas from motion-blurred videos," Computer Vision and Pattern Recognition (CVPR), pp. 2424-2431, 2010.
  10. S. H. Cho, J. Wang, and S. Y. Lee, "Video deblurring for hand-held cameras using patch-based synthesis," ACM Transactions on Graphics (TOG) Vol. 31, No. 4, 2012.
  11. B. D. Lucas, and T. Kanade, "An iterative image registration technique with an application to stereo vision," International Joint Conferences on Artificial Intelligence (IJCAI), Vol. 81, 1981.
  12. E. Mair, G. Hager, D. Burschka, M. Suppa, and G. Hirzinger, "Adaptive and generic corner detection based on the accelerated segment test," Computer Vision-ECCV 2010. Springer Berlin Heidelberg, pp. 183-196. 2010.
  13. D. G. Lowe, "Object recognition from local scale-invariant features," The proceedings of the seventh IEEE international conference on computer vision, Vol. 2. 1999.
  14. N. D. Narvekar, and L. J. Karam, "A no-reference image blur metric based on the cumulative probability of blur detection (CPBD)," IEEE Transactions on Image Processing, Vol. 20, No. 9, pp. 2678-2683, 2011. https://doi.org/10.1109/TIP.2011.2131660
  15. C. W. Son, and H. M. Park, "Fast Multiple-Image-Based Deblurring Method," Journal of The Institute of Electronics Engineers of Korea, Vol. 49-SP, No. 4, pp. 49-57, 2012.