Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Modified Seam Finding Algorithm based on Saliency Map to Generate 360 VR Image

360 VR 영상 제작을 위한 Saliency Map 기반 Seam Finding 알고리즘

  • 한현덕 (세종대학교 전자정보통신공학과) ;
  • 한종기 (세종대학교 전자정보통신공학과)
  • Received : 2019.08.16
  • Accepted : 2019.10.01
  • Published : 2019.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

The cameras generating 360 VR image are too expensive to be used publically. To overcome this problem, we propose a way to use smart phones instead of VR camera, where more than 100 pictures are taken by smart phone and are stitched into a 360 VR image. In this scenario, when moving objects are in some of the pictures, the stitched 360 VR image has various degradations, for example, ghost effect and mis-aligning. In this paper, we proposed an algorithm to modify the seam finding algorithms, where the saliency map in ROI is generated to check whether the pixel belongs to visually salient objects or not. Various simulation results show that the proposed algorithm is effective to increase the quality of the generated 360 VR image.

현재 360 VR 이미지를 만들어주는 카메라들은 상당히 고가이기에 사람들이 손쉽게 사용할 순 없는 상황이다. 이를 해결하기 위해 휴대 전화의 카메라를 이용해 100여 장의 사진을 360° 촬영을 한 후 Image stitching으로 360 VR 영상을 얻고자 한다. 기존의 장비는 한 번에 360℃ 촬영으로 VR 영상을 만들어내는 반면 휴대 전화를 이용하여 촬영할 경우 영상마다 시차가 생기게 된다. 이로 인해 움직이는 물체가 있는 경우 물체가 여러 장의 영상에서 나타나는 원하지 않는 상황이 생기게 되고 Seam이 물체를 관통하여 부자연스러운 결과 영상을 얻게 된다. 본 논문에서는 시각적으로 두드러지는 물체를 판별할 수 있는 Saliency map을 이용한 Seam finder 알고리즘을 통해 개선된 결과 영상을 얻을 수 있음을 확인했다.

Keywords

References

  1. Matthew Brown and David G. Lowe, "Automatic Panoramic Image Stitching using Invariant Features," International Journal of Computer Vision, Vol. 74, No. 1, pp. 59-73, 2007 https://doi.org/10.1007/s11263-006-0002-3
  2. Cheng-Ming Huang, Shu-Wei Lin, Jyun-Hong Chen, "Efficient Image Stitching of Continuous Image Sequence With Image and Seam Selections," IEEE Sensors Journal, Vol 15, no. 10, pp. 5910-5918, October 2015. https://doi.org/10.1109/JSEN.2015.2449879
  3. H. Bay, T.Tuytelaars, and L. Gool, "SURF : Speeded Up Robust Features," In: A.Leonardis, H. Bischof, A. Pinz(Eds.): ECCV 2006, Part 1, LNCS 3951, pp 404-417, Springer 2006
  4. E. Rublee, V. Rabaud, K. Konolige, G. Bradski, "ORB: An efficient alternative to SIFT or SURF," Proc. IEEE Int. Conf. Comput. Vis. (ICCV), pp. 2564-2571, Nov. 2011
  5. C. Bore, "The Algebra of Video Mixing: Alpha Blending and Color Keying," BORES Signal Processing, 2014
  6. M. Afifi, K. F. Hussain, "MPB: A modified Poisson blending technique," Computational Visual Media, vol. 1, no. 4, pp.331-341, 2015 https://doi.org/10.1007/s41095-015-0027-z
  7. A. Levin, A. Zomet, S. Peleg, and Y. Weiss, "Seamless image stitching in the gradient domain," Proc. of Eur. Conf. Comput. Vis., Prague, Czech Republic, Vol. 4, pp 377-389, May 2004
  8. Y.Wan, Z.Miao, "Automatic panorama image mosaic and ghost eliminating," Multimedia and Expo 2008 IEEE International Conference on, pp 509-516, 2001
  9. B.-S.Kim, K.-A.Choi, W.-J.Park, S.-W.Kim, and S.-J.Ko, "Content-preserving video stitching method for multi-camera systems," IEEE Transactions on Consumer Electronics, vol. 63, no. 2, 2017
  10. J. Yoon and D. Lee, "Real-time video stitching using camera path estimation and homography refinement," Symmetry, vol. 10, no. 1, pp 4, 2017 https://doi.org/10.3390/sym10010004
  11. W. Jiang and J. Gu, "Video stitching with spatial-temporal content-preserving warping," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 42-48, 2015
  12. Jun Pan, Mi Wang, Deren Li, and Jonathan Li, "Automatic Generation of Seamline Network Using Area Voronoi Diagrams With Overlap," IEEE Transactions on Geoscience and Remote Sensing , Vol. 47, Issue. 6, pp 1737-1744, April 2009 https://doi.org/10.1109/TGRS.2008.2009880
  13. V. C. S. Chew, F. -L. Lian, "Panorama stitching using overlap area weighted image plan projection and dynamic programming for visual localization," Proc. IEEE/ASME Int. Conf. Adv. Intell. Mechatron. (AIM), Vol. 20, No. 5, pp 728-737, September 2015
  14. Yuri Y. Boykov and Marie-Pierre Jolly, "Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images," International Conference on Computer Vision, Vol. 1, pp.105-112, July 2001
  15. R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, "Frequencytuned Salient Region Detection," IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2009
  16. J. L. Crowley, O. Riff, and J. H. Piater, "Fast computation of characteristic scale using a half octave pyramid," International Conference on Scale-Space theories in Computer Vision, 2003
  17. Cao Congjun, Sun Jing, "Study on Color Space Conversion between CMYK and CIE L*a*b* Based on Generalized Regression Neural Network," International Conference on Computer Science and Software Engineering, 2008
  18. Smith, Thomas; Guild, John, "The C.I.E. colorimetric standards and their use," Transactions of the Optical Society 33 (3), pp 73-134 1931-32 https://doi.org/10.1088/1475-4878/33/3/301
  19. Speranskaya, N.I., "Determination of spectrum color coordinates for 27 normal observers," Optics and Spectroscopy 7, 1959
  20. A. C. Harris, I. L. Weatherall, "Objective evaluation of colour variation in the sand-burrowing beetle Chaerodes trachyscelides White (Coleoptera : Tenebrionidae) by instrumental determination of CIELAB values," Journal of the Royal Society of New Zealand 20 (3), September 1990