Browse > Article
http://dx.doi.org/10.3837/tiis.2017.11.013

Fast image stitching method for handling dynamic object problems in Panoramic Images  

Abdukholikov, Murodjon (Department of IT Convergence Engineering, Gachon University)
Whangbo, Taegkeun (Department of IT Convergence, College of IT, Gachon University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.11, no.11, 2017 , pp. 5419-5435 More about this Journal
Abstract
The construction of panoramic images on smartphones and low-powered devices is a challenging task. In this paper, we propose a new approach for smoothly stitching images on mobile phones in the presence of moving objects in the scene. Our main contributions include handling moving object problems, reducing processing time, and generating rectangular panoramic images. First, unique and robust feature points are extracted using fast ORB method and a feature matching technique is applied to match the extracted feature points. After obtaining good matched feature points, we employ the non-deterministic RANSAC algorithm to discard wrong matches, and the hommography transformation matrix parameters are estimated with the algorithm. Afterward, we determine precise overlap regions of neighboring images and calculate their absolute differences. Then, thresholding operation and noise removal filtering are applied to create a mask of possible moving object regions. Sequentially, an optimal seam is estimated using dynamic programming algorithm, and a combination of linear blending with the mask information is applied to avoid seam transition and ghosting artifacts. Finally, image-cropping operation is utilized to obtain a rectangular boundary image from the stitched image. Experiments demonstrate that our method is able to produce panoramic images quickly despite the existence of moving objects.
Keywords
Image stitching; moving object; moving object detection; difference of overlap regions; image cropping; ghosting artifacts;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Abdukholikov Murodjon, Taeg-keun Whangbo, "A method for manipulating moving objects in panoramic image stitching," in Proc. of 2017 International conference on Emerging Trends & Innovation in ICT(ICEI), India, 3-5 Feb., 2017.
2 Yingen Xiong, Nokia Research Center, "Eliminating ghosting artifacts for panoramic images," 2013 IEEE International Symposium on Multimedia, pp.432-437, 2009.
3 Ebtsam Adel, Mohammed Elmogy, Hazem Elbakry, "Image stitching system based on ORB feature based technique and compensation blending,"( IJACSA) International Journal of Advanced Computer Science and Applications, vol. 6, no. 9, 2015.
4 D. G. Lowe, "Distinctive image features from scales-invariant keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.   DOI
5 Bay, Herbert, Andreas Ess, Tinne Tuytelaars, and Luc Van Gool. Speed-Up Robust Features (SURF) Comput. Vis. Image Underst. -New York, NY, USA: Elsevier Science Inc., vol. 110, no. 3, pp. 346-359. -ISSN: 1077-3142, jun. 2008.   DOI
6 Rublee, Ethan, Vincent Rabaud, Kurt Konolige, and Gary Bradski. ORB: An Efficient Alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision. - Washington, DC, USA: IEEE Computer Society, pp. 2564-2571. - ISBN: 978-1-4577-1101-5, 2011.
7 Wuxia Yan, Chuancai Liu, Wei Luo, "Fast and low complexity image stitching method on mobile phones," in Proc. of Control, Automation and Information Sciences(ICCAIS), 2015 International Conference, 2015.
8 Yanli Wan and Zhenjiang Miao, "Automatic panorama image mosaic and ghost eliminating," in Proc. of Multimedia and Expo, 2008 IEEE International Conference on., IEEE, pp.945-948, 2008.
9 Jisung Yoo, Sung Soo Hwang, Seong Dae Kim, Myung Seok Ki, Jihun Cha,"Regional Linear Warping for Image Stitching with Dominant Edge Extraction," KSII Transactions on Internet and Information Systems, vol. 7, no. 10, pp. 2464-2478, 2013,   DOI
10 Jun-Wei Hsieh, "Fast stitching algorithm for moving object detection and mosaic construction," Image and Vision Computing, vol. 22, pp 291-306, 2004,   DOI
11 Chung-Ching Lin, Sharath Pankanti, Karthikeyan Natesan Ramamurthy, and Aleksandr Y. Aravkin, "Adaptive As-Natural-As-Possible Image Stitching," in Proc. of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
12 Qingpeng Chai, Shiguang Liu, "Shape-optimizing hybrid warping for image stitching," in Proc. of 2016 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2016.
13 Xiaomin Ma, Ding Liu, Jian Zhang, Jing Xin, "A fast affine-invariant features for image stitching under large viewpoint changes," Neurocomputing, vol. 151, no. 3, pp. 1430-1438, 2015.   DOI
14 M. Kazhdan and H.Hoppe, "Streaming multigrid for gradient-domain operations on large images," ACM Trans. Graph, vol. 27, no. 3, pp. 1-10, 2008.
15 A. Alahi, R. Ortiz, P. Vandergheynst, "FREAK: Fast Retina KeyPoint," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 510-517, 2012.
16 Yu-jing Zhao, Zhe-qi Lu and Yang-ke Liu, "Video image stitching based on moving object detection and motion prediction compensation," Image and Signal Processing(CISP), 2010 3 rd International Congress, vol. 6, pp. 2885-2889, 2010.
17 J. Davis, "Mosaics of scenes with moving objects," in Proc. of CVPR 98: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, USA: IEEE Computer Society, p.354, 1998.
18 Alec Mills, Gregory Dudek, "Image stitching with dynamic elements," Image and Vision Computing, vol. 27, issue 10, pp. 1593-1602, 2009.   DOI
19 Edsger W. Dijkstra, "A note on two problems in connexion with graphs," Numerische Matematik, pp. 269-271, 1959.
20 K. Ram Prabhakar, R. Venkatesh Babu "Ghosting-free multi-exposure image fusion in gradient domain" in Proc. of Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference,
21 Yingen Xiong, Kari Pulli "Gradient Domain Image blending and Implementation on Mobile Devices," in Proc. of MobiCase '09: The First Annual International Conference on Mobile Computing, Applications, and Services', USA, 2009.
22 P. Perez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Trans Graph.,vol. 22, no. 3, pp. 313-318, 2003,   DOI
23 J. Jia, J. Sun, C.-K. Tang, and H.-Y. Shum, "Drag-and-drop pasting," in SIGGRAPH '06: ACM SIGGRAPH 2006 Papers, New York, NY, USA: ACM, pp.631-637, 2006.
24 Y. Boykov, O. Veksler and R. Zabih. "Fast approximate energy minimization via graph cuts," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, 2001,   DOI
25 P.J. Burt, E.H. Adelson, "Multiresolution spline with application to image mosaics," ACM Transactions on Graphics, vol. 2, no. 4, pp. 217-236, 1983,   DOI
26 S. Ha, H. Koo, S. Lee, N. Cho and S. Kim, "Panorama mosaic optimization for mobile camera systems," IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1217-1225, Nov. 2007,   DOI
27 Xiaoqing Yin, Weili Li, Bin Wang, Yu Liu and Maojun Zhang, "A Novel Video Stitching Method for Multi-Camera Surveillance Systems," KSII Transactions on Internet and Information Systems, vol. 8, no. 10, pp.3538-3556, 2014.   DOI
28 S. Ha, H. Koo, S. Lee, N. Cho and S. Kim, "Panorama mosaic optimization for mobile camera systems," IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1217-1225, Nov. 2007,   DOI
29 A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," ACM SIGGRAPH, pp. 341-346, 2001,
30 S. N. Sinha, J. Michael Frahm, M. Pollefeys, and Y. Genc. "GPU-based video feature tracking and matching," Technical report, In workshop on Edge Computing Using New Commodity Architectures, 2006,
31 E. Rosten and and T. Drummond. "Machine Learning for high speed corner detection.," In European Conference on Computer Vision, vol. 1, 2006,
32 M. Calonder, V. Lepetit, C. Streacha, and P. Fua. "BRIEF: Binary robust independent elementary features," in Proc. of European Conference on Computer Vision, 2010.
33 P. L. Rosin, "Measuring corner properties," Computer Vision and Image Understanding, vol. 73, no. 2, pp. 291-307, 1999.   DOI
34 Panorama Maker, https://panorama-maker.en.softonic.com
35 M. Brown and D. G. Lowe, "Automatic panoramic image stitching using invariant features," International Journal of Computer Vision, vol. 74, no. 1, pp. 59-73, 2007.   DOI