Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Stabilization Technique for 360 VR Video Sequences

360 VR 영상용 흔들림 제거 기술

  • Kim, Geun-Bae (Sejong University, Dept. of Electrical Engineering) ;
  • Lee, Jae-Yung (Sejong University, Dept. of Electrical Engineering) ;
  • Han, Jong-Ki (Sejong University, Dept. of Electrical Engineering)
  • 김근배 (세종대학교 전자정보통신공학과) ;
  • 이재영 (세종대학교 전자정보통신공학과) ;
  • 한종기 (세종대학교 전자정보통신공학과)
  • Received : 2017.10.12
  • Accepted : 2017.10.27
  • Published : 2017.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose an efficient stabilization algorithm for 360 VR sequence, where the geometric information and the relationship between various projection formats have been utilized. The proposed scheme consists of '1st stitching', 'extraction of wide angle picture from VR data', 'stabilization', and '2nd stitching'. The simulation results show that the proposed method outperforms the conventional algorithms in the viewpoint of image quality for end-user.

본 논문에서는 360 VR 비디오 신호에 적합한 흔들림 제거 기술을 제안한다. 이를 위해서 VR 방송 시스템의 구성 요소들인 스티칭 및 투영포맷(projection format) 등의 특성을 분석하고, 각 단계에서 확보되는 영상 정보들의 기하학적인 특성들을 분석하였다. 이런 정보들을 활용하여 VR 영상 신호용 흔들림 제거 기술을 개발함으로써, 이러한 고려없이 2D 비디오 신호용으로 개발되어온 기존의 흔들림 제거 기술보다 우수한 성능을 제공하는 흔들림 제거 모듈을 구현할 수 있었다. 본 논문에서 제안된 알고리즘의 성능을 기존의 Autopano의 성능과 비교한 결과 물체의 이동 궤적의 흔들림이 눈에 띄게 줄어들었고, 흔들림이 존재하는 비디오 신호의 연속된 프레임들간의 RMSE가 26.04와 5.78 이였던 것이 제안되는 기술로 각각 7.33과 4.38까지 줄일 수 있었다.

Keywords

References

  1. C. Kim, "A Comparative Study for Virtual Reality $360^{\circ}$ Contents Shooting Equipments based on Real World," Journal of Broadcast Engineering, Vol. 21, No. 5, pp 714-725, Sept. 2016. https://doi.org/10.5909/JBE.2016.21.5.714
  2. Silicon Valley Reports, https://jointventure.org/publications/institute-publications/1523-2017-silicon-valley-index (accessed Sept. 5 2017)
  3. Global Tech. Korea Reports, http://www.gtonline.or.kr/kor/gtbase/all/data/policy/dataView.docPage=1&sch_gtbase_cd=&sch_national_cd=&sch_tech_1st_gbn_cd=&sch_tech_2nd_gbn_cd=&searchField= &keyword= &data_sid=223762 (accessed Sept. 5 2017)
  4. M. Grundmann, V. Kwatra, and I. Essa, "Auto-directed Video Stabilization with Robust L1 Optimal Camera Paths," Computer Vision and Pattern Recognition (CVPR), Colorado USA, June 2011.
  5. A. Litvin, J. Konrad, and W. C. Karl, "Probabilistic Video Stabilization using Kalman Filtering and Mosaicking," IS&T/SPIE Symposium on Electronic Imaging, Image and Video Communications and Proc., Santa Clara, CA, USA, Jan. 2003.
  6. K.-Y. Lee, Y.-Y. Chuang, B.-Y. Chen, and M. Ouhyoung, "Video Stabilization using Robust Feature Trajectories," IEEE 12th International Conference on Computer Vision (ICCV), Kyoto, Japan, 2009.
  7. Y. Ye, E. Alshina, and J. Boyce, "Algorithm Descriptions of Projection Format Conversion and Video Quality Metrics in 360 Lib," JVET-E1003, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 5th Meeting: Geneva, CH, 12-20 January 2017.
  8. https://jvet.hhi.fraunhofer.de/svn/svn_360Lib/tags/360Lib-2.0
  9. C. Harris and M. Stephens, "A Combined Corner and Edge Detector," Fourth Alvey Vision Conference, pp. 147-151, 1988.
  10. B. K. P. Horn and B. G. Schunck, "Determining Optical Flow," Artificial Intelligence, Vol. 17, Issues 1-3, pp. 185-203, August 1981. https://doi.org/10.1016/0004-3702(81)90024-2
  11. B. Widrow, and S. D. Stearns, Adaptive Signal Processing, Prentice Hall, 1985.
  12. J. Dong and H. Liu, "Video Stabilization for Strict Real-Time Applications," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 27, Issue 4, pp. 716-724, April 2017. https://doi.org/10.1109/TCSVT.2016.2589860
  13. W. G. Aguilar, T. Abaroa, R. Fuertes, A. Ibarra, and D. Loza, "Video Stabilization System without Phantom Movements for Cameras Onboard Rotocrafts," 2017 1st Latin American Conference on Intelligent Transportation Systems (ITS LATAM), Bogota, Colombia, 2017.