Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.1.20

Panorama Image Stitching Using Sythetic Fisheye Image  

Kweon, Hyeok-Joon (Department of Electronics Engineering, Chungnam National University)
Cho, Donghyeon (Department of Electronics Engineering, Chungnam National University)
Publication Information
Journal of Broadcast Engineering / v.27, no.1, 2022 , pp. 20-30 More about this Journal
Abstract
Recently, as VR (Virtual Reality) technology has been in the spotlight, 360° panoramic images that can view lively VR contents are attracting a lot of attention. Image stitching technology is a major technology for producing 360° panorama images, and many studies are being actively conducted. Typical stitching algorithms are based on feature point-based image stitching. However, conventional feature point-based image stitching methods have a problem that stitching results are intensely affected by feature points. To solve this problem, deep learning-based image stitching technologies have recently been studied, but there are still many problems when there are few overlapping areas between images or large parallax. In addition, there is a limit to complete supervised learning because labeled ground-truth panorama images cannot be obtained in a real environment. Therefore, we produced three fisheye images with different camera centers and corresponding ground truth image through carla simulator that is widely used in the autonomous driving field. We propose image stitching model that creates a 360° panorama image with the produced fisheye image. The final experimental results are virtual datasets configured similar to the actual environment, verifying stitching results that are strong against various environments and large parallax.
Keywords
360-panorama; Fisheye; Stitching; Carla simulator;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. ORB: An efficient alternative to SIFT or SURF. In International conference on computer vision, pp. 2564-2571 , November 2011.
2 Rosten, E., & Drummond, T. Machine learning for high-speed corner detection. In European conference on computer vision, pp. 430-443, May 2006.
3 Lowe, D. G. Distinctive image features from scale-invariant keypoints. In International journal of computer vision, 60(2), 91-110, 2004.   DOI
4 Burt, Peter J., and Edward H. Adelson. "A multiresolution spline with application to image mosaics." In ACM Transactions on Graphics (TOG) 2.4, 217-236, 1983 .   DOI
5 Perez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. In ACM SIGGRAPH, pp. 313-318, 2003 .
6 Lin, W. Y., Liu, S., Matsushita, Y., Ng, T. T., & Cheong, L. F. Smoothly varying affine stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 345-352, June 2011
7 Zaragoza, J., Chin, T. J., Brown, M. S., & Suter, D. As-projective-as-possible image stitching with moving DLT. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2339-2346, 2013.
8 Lee, K. Y., & Sim, J. Y. Warping residual based image stitching for large parallax. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8198-8206, 2020.
9 Harris, C., & Stephens, M. A combined corner and edge detector. In Alvey vision conference, Vol. 15, No. 50, pp. 10-5244, August 1988.
10 Gao, J., Kim, S. J., & Brown, M. S. Constructing image panoramas using dual-homography warping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 49-56, June 2011.
11 Wang, L., Yu, W., & Li, B. (2020, June). Multi-scenes image stitching based on autonomous driving. In IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Vol. 1, pp. 694-698, June 2020.
12 Lin, K., Jiang, N., Cheong, L. F., Do, M., & Lu, J. Seagull: Seam-guided local alignment for parallax-tolerant image stitching. In European conference on computer vision, pp. 370-385, October 2016.
13 Hejazifar, H., & Khotanlou, H. Fast and robust seam estimation to seamless image stitching. In Signal, Image and Video Processing, 12(5), 885-893, 2018.   DOI
14 Shi, Z., Li, H., Cao, Q., Ren, H., & Fan, B. An image mosaic method based on convolutional neural network semantic features extraction. In Journal of Signal Processing Systems, 92(4), 435-444, 2020.   DOI
15 Nie, L., Lin, C., Liao, K., Liu, M., & Zhao, Y. A view-free image stitching network based on global homography. In Journal of Visual Communication and Image Representation, 73, 102950, 2020.   DOI
16 Song, D. Y., Um, G. M., Lee, H. K., & Cho, D. End-to-End Image Stitching Network via Multi-Homography Estimation. In IEEE Signal Processing Letters, 28, 763-767, 2021.   DOI
17 Bay, H., Tuytelaars, T., & Van Gool, L. Surf: Speeded up robust features. In European conference on computer vision, pp. 404-417, May 2006.
18 Berenguel-Baeta, B., Bermudez-Cameo, J., & Guerrero, J. J. OmniSCV: An Omnidirectional Synthetic Image Generator for Computer Vision. In Sensors, 20(7), 2066, 2020.   DOI
19 Scaramuzza, D., Martinelli, A., & Siegwart, R. (2006, October). A toolbox for easily calibrating omnidirectional cameras. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5695-5701, October 2006.
20 Zhu, Hancheng, et al. MetaIQA: Deep meta-learning for no-reference image quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
21 Fischler, M. A., & Bolles, R. C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. In Communications of the ACM, 24(6), 381-395, 1981.   DOI
22 Chang, C. H., Sato, Y., & Chuang, Y. Y. Shape-preserving half-projective warps for image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254-3261, 2014.
23 Zhang, F., & Liu, F. Parallax-tolerant image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262-3269, 2014.
24 Hoang, V. D., Tran, D. P., Nhu, N. G., & Pham, V. H. Deep feature extraction for panoramic image stitching. In Asian Conference on Intelligent Information and Database Systems, pp. 141-151, March 2020.
25 Li, J., Zhao, Y., Ye, W., Yu, K., & Ge, S. Attentive Deep Stitching and Quality Assessment for 360° Omnidirectional Images. In IEEE Journal of Selected Topics in Signal Processing, 14(1), 209-221, 2019.   DOI
26 Sekkat, A. R., Dupuis, Y., Vasseur, P., & Honeine, P. The omniscape dataset. In IEEE International Conference on Robotics and Automation (ICRA), pp. 1603-1608, May 2020.
27 Mittal, Anish, Rajiv Soundararajan, and Alan C. Bovik. Making a "completely blind" image quality analyzer. In IEEE Signal Processing Letters (SPL), pp. 209-212. 20.3 (2012)   DOI
28 Liu, F., Gleicher, M., Jin, H., & Agarwala, A. Content-preserving warps for 3D video stabilization. In ACM Transactions on Graphics (ToG), 28(3), 1-9, 2009.