Browse > Article
http://dx.doi.org/10.12989/sss.2021.28.5.661

Feature-based image stitching for panorama construction and visual inspection of structures  

Cheng, Kai (Department of Disaster Mitigation for Structures, Tongji University)
Shan, Jiazeng (Department of Disaster Mitigation for Structures, Tongji University)
Liu, Yuwen (Department of Disaster Mitigation for Structures, Tongji University)
Publication Information
Smart Structures and Systems / v.28, no.5, 2021 , pp. 661-673 More about this Journal
Abstract
This study presents a feature-based image stitching method with multi-level constraint criterion for panorama construction and visual inspection of building structures. The comparison of global view and local resolution over building exterior is discussed regarding practical implementation. An inspection-oriented methodology framework with optimized inlier distribution is designed for generating a feasible and reliable building panorama by using ordinary optic images. Two illustrative examples, including an earthquake-damaged masonry wall and a high-rise building with stone curtain walls, are experimentally investigated. The severely developed structural crack is fully mapped with stitched image and extracted in preparation for further quality evaluation. The curtain wall of the high-rise building is successfully constructed by using UAV-based images. The panorama quality is further compared with commercial stitching software and several improvements are illustrated in the particular case. In addition, the reliability of the proposed feature-based stitching approach is parametrically studied with different setups of input images.
Keywords
computer vision; image stitching; multi-level constraint criterion; structural health monitoring; visual inspection;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Adobe Photoshop (2006), Create and edit a panorama in Photoshop, Adobe Photoshop, San Jose, USA. https://helpx.adobe.com/photoshop/using/create-panoramicimages-photomerge.html
2 Akbar, M.A., Qidwai, U. and Jahanshahi, M.R. (2018), "An evaluation of image-based structural health monitoring using integrated unmanned aerial vehicle platform", Struct. Control Health Monitor., 26(1), e2276. https://doi.org/10.1002/stc.2276   DOI
3 Bang, S., Kim, H. and Kim, H. (2017), "UAV-based automatic generation of high-resolution panorama at a construction site with a focus on preprocessing for image stitching", Automat. Constr., 84, 70-80. https://doi.org/10.1016/j.autcon.2017.08.031   DOI
4 Sun, Y., Zhao, L., Huang, S., Yan, L. and Dissanayake, G. (2014), "L2-SIFT: SIFT feature extraction and matching for large images in large-scale aerial photogrammetry", ISPRS J. Photogram. Remote Sens., 91, 1-16. https://doi.org/10.1016/j.isprsjprs.2014.02.001   DOI
5 Torr, P.H.S. and Zisserman, A. (2000), "MLESAC: a new robust estimator with application to estimating image geometry", Comput. Vis. Image Underst., 78(1), 138-156. https://doi.org/10.1006/cviu.1999.0832   DOI
6 Xiang, T., Xia, G. and Zhang, L. (2019), "Mini-unmanned aerial vehicle-based remote sensing: techniques, applications, and prospects", IEEE Geosci. Remote Sens. Magaz., 7(3), 29-63. https://doi.org/10.1109/MGRS.2019.2918840   DOI
7 Ye, X.W., Dong, C.Z. and Liu, T. (2016), "Image-based structural dynamic displacement measurement using different multi-object tracking algorithms", Smart Struct. Syst., Int. J., 17(6), 935-956. https://doi.org/10.12989/sss.2016.17.6.935   DOI
8 Lai, Z., Alzugaray, I., Chli, M. and Chatzi, E. (2020), "Full-field structural monitoring using event cameras and physics-informed sparse identification", Mech. Syst. Signal Process., 145, 106905. https://doi.org/10.1016/j.ymssp.2020.106905   DOI
9 Brown, M. and Lowe, D.G. (2007), "Automatic panoramic image stitching using invariant features", Int. J. Comput. Vis., 74(1), 59-73. https://doi.org/10.1007/s11263-006-0002-3   DOI
10 Jahanshahi, M.R., Masri, S.F. and Sukhatme, G.S. (2011), "Multi-image stitching and scene reconstruction for evaluating defect evolution in structures", Struct. Health Monitor., 10(6), 643-657. https://doi.org/10.1177/1475921710395809   DOI
11 Lowe, D.G. (2004), "Distinctive image features from scale-invariant keypoints", Int. J. Comput. Vis., 60(2), 91-110. https://doi.org/10.1023/B:VISI.0000029664.99615.94   DOI
12 Luo, J. and Oubong, G. (2009), "A comparison of SIFT, PCA-SIFT and SURF", Int. J. Image Process., 3(4), 143-152.
13 Moore, M., Phares, B.M., Graybeal, B., Rolander, D. and Washer, G. (2001), "Reliability of Visual Inspection for Highway Bridges", Final Rep. No. FHWA-RD-01-020, Federal Highway Administration (FHWA).
14 Rousseeuw, P.J. (1984), "Least median of squares regression", J. Am. Statist. Assoc., 79(388), 871-880.   DOI
15 Jung, H.J., Lee, J.H., Yoon, S. and Kim, I.H. (2019), "Bridge inspection and condition assessment using unmanned aerial vehicles (UAVs): major challenges and solutions from a practical perspective", Smart Struct. Syst., Int. J., 24(5), 669-681. https://doi.org/10.12989/sss.2019.24.5.669   DOI
16 Nie, L., Lin, C., Liao, K., Liu, S. and Zhao, Y. (2021), "Unsupervised deep image stitching: reconstructing stitched features to images", IEEE Transact. Image Process., 30, 6184-6197. https://doi.org/10.1109/TIP.2021.3092828   DOI
17 Rosten, E., Porter, R. and Drummond, T. (2010), "Faster and better: a machine learning approach to corner detection", IEEE Transact. Pattern Anal. Mach. Intell., 32(1), 105-119. https://doi.org/10.1109/TPAMI.2008.275   DOI
18 Agdas, D., Rice, J.A., Martinez, J.R. and Lasa, I.R. (2016), "Comparison of visual inspection and structural-health monitoring as bridge condition assessment methods", J. Perform. Constr Facil., 30(3), 04015049. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000802   DOI
19 VLFeat - Home (2008), VLFeat: An open and portable library of computer vision algorithms, Vedaldi A. and Fulkerson B. http://www.vlfeat.org
20 Choi, J., Yeum, C.M., Dyke, S.J. and Jahanshahi, M.R. (2018), "Computer-aided approach for rapid post-event visual evaluation of a building facade", Sensors, 18(9), 3017. https://doi.org/10.3390/s18093017   DOI
21 Schonberger, J.L. and Frahm, J. (2016), "Structure-from-motion revisited", Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Nevada, USA, June.
22 Hoskere, V., Park, J.W., Yoon, H. and Spencer, B.F. (2019), "Vision-based modal survey of civil infrastructure using unmanned aerial vehicles", J. Struct. Eng., 145(7), 04019062. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002321   DOI
23 Fischler, M.A. and Bolles, R.C. (1981), "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography", Commun. ACM, 24(6), 381-395. https://doi.org/10.1145/358669.358692   DOI
24 Harris, C.G. and Stephens, M. (1988), "A combined corner and edge detector", Alvey vision conference, Manchester, UK, September.
25 Herrmann, C., Wang, C., Bowen, R.S., Keyder, E., Krainin, M., Liu, C. and Zabih, R. (2018), "Robust image stitching with multiple registrations", Proceedings of European Conference on Computer Vision, Munich, Germany, September.
26 Iacovino, C., Ditommaso, R., Ponzo, F.C. and Limongelli, M.P. (2018), "The interpolation evolution method for damage localization in structures under seismic excitation", Earthq. Eng. Struct. Dyn., 47(10), 2117-2136. https://doi.org/10.1002/eqe.3062   DOI
27 Lee, K.Y. and Sim, J.Y. (2020), "Warping residual based image stitching for large parallax", 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, Washington, USA, June.
28 Microsoft: Image Composite Editor (2008), Image Composite Editor - Microsoft Research, Microsoft, Redmond, USA. https://www.microsoft.com/enus/research/product/computational-photographyapplications/image-composite-editor/
29 Bay, H., Ess, A., Tuytelaars, T. and Van Gool, L. (2008), "Speeded-up robust features (SURF)", Comput. Vis. Image Underst., 110(3), 346-359. https://doi.org/10.1016/j.cviu.2007.09.014   DOI
30 Jiang, X., Ma, J., Jiang, J. and Guo, X. (2020), "Robust feature matching using spatial clustering with heavy outliers", IEEE Transactions on Image Processing, 29, 736-746.   DOI
31 Spencer, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030   DOI
32 Hsu, T.Y., Pham, Q.V., Chao, W.C. and Yang, Y.S. (2020), "Post-earthquake building safety evaluation using consumer-grade surveillance cameras", Smart Struct. Syst., Int. J., 25(5), 531-541. https://doi.org/10.12989/sss.2020.25.5.531   DOI
33 Yeum, C.M., Choi, J. and Dyke, S.J. (2017), "Autonomous image localization for visual inspection of civil infrastructure", Smart Mater. Struct., 26(3), 035051. https://doi.org/10.1088/1361-665X/aa510e   DOI
34 Zhou, W., Bovik, A.C., Sheikh, H.R. and Simoncelli, E.P. (2004), "Image quality assessment: from error visibility to structural similarity", IEEE Transact. Image Process., 13(4), 600-612. https://doi.org/10.1109/TIP.2003.819861   DOI
35 Sun, L., Shang, Z., Xia, Y., Bhowmick, S. and Nagarajaiah, S. (2020), "Review of bridge structural health monitoring aided by big data and artificial intelligence: from condition assessment to damage detection", J. Struct. Eng., 146(5), 04020073. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002535   DOI
36 OpenCV (2011), OpenCV: Images stitching, OpenCV team, USA. https://docs.opencv.org/master/d1/d46/group__stitching.html
37 Rublee, E., Rabaud, V., Konolige, K. and Bradski, G. (2011), "ORB: An efficient alternative to SIFT or SURF", Proceedings of 2011 International Conference on Computer Vision, Barcelona, Spain, November. https://doi.org/10.1109/ICCV.2011.6126544   DOI
38 Strauss, A., Bien, J., Neuner, H., Harmening, C., Seywald, C., Osterreicher, M., Voit, K., Pistone, E., Spyridis, P. and Bergmeister, K. (2020), "Sensing and monitoring in tunnels testing and monitoring methods for the assessment of tunnels", Struct. Concrete, 21(4), 1356-1376. https://doi.org/10.1002/suco.201900444   DOI
39 Szeliski, R. (2007), Image Alignment And Stitching: A Tutorial, Now Publishers Inc., Boston, MA, USA.
40 Wu, R.T. and Jahanshahi, M.R. (2020), "Data fusion approaches for structural health monitoring and system identification: past, present, and future", Struct. Health Monitor., 19(2), 552-586. https://doi.org/10.1177/1475921718798769   DOI
41 Yuan, M., Lei, T., Liu, X. and Li, S. (2019), "Fast image stitching of unmanned aerial vehicle remote sensing image based on SURF algorithm", Proceedings of the 11th International Conference on Digital Image Processing (ICDIP 2019), Guangzhou, China, May.