Browse > Article
http://dx.doi.org/10.5909/JBE.2020.25.6.882

Image Stitching focused on Priority Object using Deep Learning based Object Detection  

Rhee, Seongbae (Department of Electronic Engineering, Kyung Hee University)
Kang, Jeonho (Department of Electronic Engineering, Kyung Hee University)
Kim, Kyuheon (Department of Electronic Engineering, Kyung Hee University)
Publication Information
Journal of Broadcast Engineering / v.25, no.6, 2020 , pp. 882-897 More about this Journal
Abstract
Recently, the use of immersive media contents representing Panorama and 360° video is increasing. Since the viewing angle is limited to generate the content through a general camera, image stitching is mainly used to combine images taken with multiple cameras into one image having a wide field of view. However, if the parallax between the cameras is large, parallax distortion may occur in the stitched image, which disturbs the user's content immersion, thus an image stitching overcoming parallax distortion is required. The existing Seam Optimization based image stitching method to overcome parallax distortion uses energy function or object segment information to reflect the location information of objects, but the initial seam generation location, background information, performance of the object detector, and placement of objects may limit application. Therefore, in this paper, we propose an image stitching method that can overcome the limitations of the existing method by adding a weight value set differently according to the type of object to the energy value using object detection based on deep learning.
Keywords
Image Stitching; Parallax Distortion; Seam Optimization; Object Detection; Priority Object;
Citations & Related Records
Times Cited By KSCI : 6  (Citation Analysis)
연도 인용수 순위
1 Im, Jiheon, et al. "Images Grouping Technology based on Camera Sensors for Efficient Stitching of Multiple Images." Journal of Broadcast Engineering, Vol.22, No.6, pp.713-723, 2017.   DOI
2 R. Szeliski, "Image Alignment and Stitching: A Tutorial." Foundations and Trends in Computer Graphics and Computer Vision, Vol. 2, No.1, 2006.
3 Rhee, Seongbae, Jeonho Kang, and Kyuheon Kim. "Local Differential Pixel Assessment Method for Image Stitching." Journal of Broadcast Engineering, Vol.24, No.5, pp.775-784, 2019   DOI
4 Wei, L. Y. U., et al. "A survey on image and video stitching." Virtual Reality & Intelligent Hardware Vol. 1, No.1 pp.55-83, 2019.   DOI
5 Lin, Kaimo, et al. "Seagull: Seam-guided local alignment for parallax-tolerant image stitching." European conference on computer vision. Springer, Cham, 2016.
6 Avidan, Shai, and Ariel Shamir. "Seam carving for content-aware image resizing." ACM SIGGRAPH 2007 papers. pp.10-es, 2007.
7 TANG, Yu; JIANG, Huiyan. "Highly efficient image stitching based on energy map". In: 2009 2nd International Congress on Image and Signal Processing IEEE, pp.1-5, 2009.
8 TANG, Yu; SHIN, Jungpil. "Image stitching with efficient brightness fusion and automatic content awareness.". International Conference on Signal Processing and Multimedia Applications (SIGMAP), pp.60-66, 2014.
9 Abdukholikov, Murodjon, and Taegkeun Whangbo. "Fast image stitching method for handling dynamic object problems in Panoramic Images." KSII Transactions on Internet & Information Systems, Vol.11, No.11, 2017.
10 Kang, Jeonho, et al. "Minimum Error Seam-Based Efficient Panorama Video Stitching Method Robust to Parallax." IEEE Access 7, pp. 167127-167140, 2019.   DOI
11 Rhee, Seongbae, Jeonho Kang, and Kyuheon Kim. "Parallax Distortion Detection and Correction Method for Video Stitching by using LDPM Image Assessment." Journal of Broadcast Engineering, Vol.25, No.5, pp.685-697, 2020.   DOI
12 He, Kaiming, et al. "Mask r-cnn." Proceedings of the IEEE international conference on computer vision, pp.2961-2969, 2017.
13 Zhang, Guofeng, et al. "Multi-viewpoint panorama construction with wide-baseline images." IEEE Transactions on Image Processing, Vol.25, No.7, pp.3099-3111, 2016.   DOI
14 Bolya, Daniel, et al. "Yolact: Real-time instance segmentation." Proceedings of the IEEE international conference on computer vision., pp.9157-9166, 2019.
15 Rublee, Ethan, et al. "ORB: An efficient alternative to SIFT or SURF." 2011 International conference on computer vision. IEEE, pp.2564-2571, 2011.