KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Fast image stitching method for handling dynamic object problems in Panoramic Images

  • Received : 2017.03.14
  • Accepted : 2017.07.20
  • Published : 2017.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

The construction of panoramic images on smartphones and low-powered devices is a challenging task. In this paper, we propose a new approach for smoothly stitching images on mobile phones in the presence of moving objects in the scene. Our main contributions include handling moving object problems, reducing processing time, and generating rectangular panoramic images. First, unique and robust feature points are extracted using fast ORB method and a feature matching technique is applied to match the extracted feature points. After obtaining good matched feature points, we employ the non-deterministic RANSAC algorithm to discard wrong matches, and the hommography transformation matrix parameters are estimated with the algorithm. Afterward, we determine precise overlap regions of neighboring images and calculate their absolute differences. Then, thresholding operation and noise removal filtering are applied to create a mask of possible moving object regions. Sequentially, an optimal seam is estimated using dynamic programming algorithm, and a combination of linear blending with the mask information is applied to avoid seam transition and ghosting artifacts. Finally, image-cropping operation is utilized to obtain a rectangular boundary image from the stitched image. Experiments demonstrate that our method is able to produce panoramic images quickly despite the existence of moving objects.

Keywords

References

  1. Abdukholikov Murodjon, Taeg-keun Whangbo, "A method for manipulating moving objects in panoramic image stitching," in Proc. of 2017 International conference on Emerging Trends & Innovation in ICT(ICEI), India, 3-5 Feb., 2017.
  2. Yingen Xiong, Nokia Research Center, "Eliminating ghosting artifacts for panoramic images," 2013 IEEE International Symposium on Multimedia, pp.432-437, 2009.
  3. Ebtsam Adel, Mohammed Elmogy, Hazem Elbakry, "Image stitching system based on ORB feature based technique and compensation blending,"( IJACSA) International Journal of Advanced Computer Science and Applications, vol. 6, no. 9, 2015.
  4. D. G. Lowe, "Distinctive image features from scales-invariant keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  5. Bay, Herbert, Andreas Ess, Tinne Tuytelaars, and Luc Van Gool. Speed-Up Robust Features (SURF) Comput. Vis. Image Underst. -New York, NY, USA: Elsevier Science Inc., vol. 110, no. 3, pp. 346-359. -ISSN: 1077-3142, jun. 2008. https://doi.org/10.1016/j.cviu.2007.09.014
  6. Rublee, Ethan, Vincent Rabaud, Kurt Konolige, and Gary Bradski. ORB: An Efficient Alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision. - Washington, DC, USA: IEEE Computer Society, pp. 2564-2571. - ISBN: 978-1-4577-1101-5, 2011.
  7. Wuxia Yan, Chuancai Liu, Wei Luo, "Fast and low complexity image stitching method on mobile phones," in Proc. of Control, Automation and Information Sciences(ICCAIS), 2015 International Conference, 2015.
  8. Yanli Wan and Zhenjiang Miao, "Automatic panorama image mosaic and ghost eliminating," in Proc. of Multimedia and Expo, 2008 IEEE International Conference on., IEEE, pp.945-948, 2008.
  9. Jun-Wei Hsieh, "Fast stitching algorithm for moving object detection and mosaic construction," Image and Vision Computing, vol. 22, pp 291-306, 2004, https://doi.org/10.1016/j.imavis.2003.09.018
  10. Jisung Yoo, Sung Soo Hwang, Seong Dae Kim, Myung Seok Ki, Jihun Cha,"Regional Linear Warping for Image Stitching with Dominant Edge Extraction," KSII Transactions on Internet and Information Systems, vol. 7, no. 10, pp. 2464-2478, 2013, https://doi.org/10.3837/tiis.2013.10.008
  11. Chung-Ching Lin, Sharath Pankanti, Karthikeyan Natesan Ramamurthy, and Aleksandr Y. Aravkin, "Adaptive As-Natural-As-Possible Image Stitching," in Proc. of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
  12. Qingpeng Chai, Shiguang Liu, "Shape-optimizing hybrid warping for image stitching," in Proc. of 2016 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2016.
  13. Xiaomin Ma, Ding Liu, Jian Zhang, Jing Xin, "A fast affine-invariant features for image stitching under large viewpoint changes," Neurocomputing, vol. 151, no. 3, pp. 1430-1438, 2015. https://doi.org/10.1016/j.neucom.2014.10.045
  14. A. Alahi, R. Ortiz, P. Vandergheynst, "FREAK: Fast Retina KeyPoint," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 510-517, 2012.
  15. Yu-jing Zhao, Zhe-qi Lu and Yang-ke Liu, "Video image stitching based on moving object detection and motion prediction compensation," Image and Signal Processing(CISP), 2010 3 rd International Congress, vol. 6, pp. 2885-2889, 2010.
  16. J. Davis, "Mosaics of scenes with moving objects," in Proc. of CVPR 98: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, USA: IEEE Computer Society, p.354, 1998.
  17. Alec Mills, Gregory Dudek, "Image stitching with dynamic elements," Image and Vision Computing, vol. 27, issue 10, pp. 1593-1602, 2009. https://doi.org/10.1016/j.imavis.2009.03.004
  18. Edsger W. Dijkstra, "A note on two problems in connexion with graphs," Numerische Matematik, pp. 269-271, 1959.
  19. M. Kazhdan and H.Hoppe, "Streaming multigrid for gradient-domain operations on large images," ACM Trans. Graph, vol. 27, no. 3, pp. 1-10, 2008.
  20. K. Ram Prabhakar, R. Venkatesh Babu "Ghosting-free multi-exposure image fusion in gradient domain" in Proc. of Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference,
  21. Yingen Xiong, Kari Pulli "Gradient Domain Image blending and Implementation on Mobile Devices," in Proc. of MobiCase '09: The First Annual International Conference on Mobile Computing, Applications, and Services', USA, 2009.
  22. P. Perez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Trans Graph.,vol. 22, no. 3, pp. 313-318, 2003, https://doi.org/10.1145/882262.882269
  23. J. Jia, J. Sun, C.-K. Tang, and H.-Y. Shum, "Drag-and-drop pasting," in SIGGRAPH '06: ACM SIGGRAPH 2006 Papers, New York, NY, USA: ACM, pp.631-637, 2006.
  24. P.J. Burt, E.H. Adelson, "Multiresolution spline with application to image mosaics," ACM Transactions on Graphics, vol. 2, no. 4, pp. 217-236, 1983, https://doi.org/10.1145/245.247
  25. S. Ha, H. Koo, S. Lee, N. Cho and S. Kim, "Panorama mosaic optimization for mobile camera systems," IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1217-1225, Nov. 2007, https://doi.org/10.1109/TCE.2007.4429204
  26. Xiaoqing Yin, Weili Li, Bin Wang, Yu Liu and Maojun Zhang, "A Novel Video Stitching Method for Multi-Camera Surveillance Systems," KSII Transactions on Internet and Information Systems, vol. 8, no. 10, pp.3538-3556, 2014. https://doi.org/10.3837/tiis.2014.10.015
  27. S. Ha, H. Koo, S. Lee, N. Cho and S. Kim, "Panorama mosaic optimization for mobile camera systems," IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1217-1225, Nov. 2007, https://doi.org/10.1109/TCE.2007.4429204
  28. A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," ACM SIGGRAPH, pp. 341-346, 2001,
  29. Y. Boykov, O. Veksler and R. Zabih. "Fast approximate energy minimization via graph cuts," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, 2001, https://doi.org/10.1109/34.969114
  30. S. N. Sinha, J. Michael Frahm, M. Pollefeys, and Y. Genc. "GPU-based video feature tracking and matching," Technical report, In workshop on Edge Computing Using New Commodity Architectures, 2006,
  31. E. Rosten and and T. Drummond. "Machine Learning for high speed corner detection.," In European Conference on Computer Vision, vol. 1, 2006,
  32. M. Calonder, V. Lepetit, C. Streacha, and P. Fua. "BRIEF: Binary robust independent elementary features," in Proc. of European Conference on Computer Vision, 2010.
  33. P. L. Rosin, "Measuring corner properties," Computer Vision and Image Understanding, vol. 73, no. 2, pp. 291-307, 1999. https://doi.org/10.1006/cviu.1998.0719
  34. M. Brown and D. G. Lowe, "Automatic panoramic image stitching using invariant features," International Journal of Computer Vision, vol. 74, no. 1, pp. 59-73, 2007. https://doi.org/10.1007/s11263-006-0002-3
  35. Panorama Maker, https://panorama-maker.en.softonic.com

Cited by

  1. Detection and Removal of Moving Object Shadows Using Geometry and Color Information for Indoor Video Streams vol.9, pp.23, 2017, https://doi.org/10.3390/app9235165