Browse > Article
http://dx.doi.org/10.12989/sss.2021.27.2.209

A new image-quality evaluating and enhancing methodology for bridge inspection using an unmanned aerial vehicle  

Lee, Jin Hwan (Department of Civil and Environmental Engineering, Korea Advanced Institute of Science and Technology)
Yoon, Sungsik (Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign)
Kim, Byunghyun (Department of Civil Engineering, University of Seoul)
Gwon, Gi-Hun (Department of Civil and Environmental Engineering, Korea Advanced Institute of Science and Technology)
Kim, In-Ho (Department of Civil Engineering, Kunsan National University)
Jung, Hyung-Jo (Department of Civil and Environmental Engineering, Korea Advanced Institute of Science and Technology)
Publication Information
Smart Structures and Systems / v.27, no.2, 2021 , pp. 209-226 More about this Journal
Abstract
This paper proposes a new methodology to address the image quality problem encountered as the use of an unmanned aerial vehicle (UAV) in the field of bridge inspection increased. When inspecting a bridge, the image obtained from the UAV was degraded by various interference factors such as vibration, wind, and motion of UAV. Image quality degradation such as blur, noise, and low-resolution is a major obstacle in utilizing bridge inspection technology based on UAV. In particular, in the field of bridge inspection where damages must be accurately and quickly detected based on data obtained from UAV, these quality issues weaken the advantage of using UAVs by requiring re-take of images through re-flighting. Therefore, in this study, image quality assessment (IQA) based on local blur map (LBM) and image quality enhancement (IQE) using the variational Dirichlet (VD) kernel estimation were proposed as a solution to address the quality issues. First, image data was collected by setting different camera parameters for each bridge member. Second, a blur map was generated through discrete wavelet transform (DWT) and a new quality metric to measure the degree of blurriness was proposed. Third, for low-quality images with a large degree of blurriness, the blind kernel estimation and blind image deconvolution were performed to enhance the quality of images. In the validation tests, the proposed quality metric was applied to material image sets of bridge pier and deck taken from UAV, and its results were compared with those of other quality metrics based on singular value decomposition (SVD), sum of gray-intensity variance (SGV) and high-frequency multiscale fusion and sort transform (HiFST) methods. It was validated that the proposed IQA metric showed better classification performance on UAV images for bridge inspection through comparison with the classification results by human perception. In addition, by performing IQE, on average, 26% of blur was reduced, and the images with enhanced quality showed better damage detection performance through the deep learning model (i.e., mask and region-based convolutional neural network).
Keywords
Unmanned Aerial Vehicle (UAV); bridge inspection; Image Quality Assessment (IQA); Image Quality Enhancement (IQE); damage detection;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Tomiczek, A.P., Bridge, J.A., Ifju, P.G., Whitley, T.J., Tripp, C.S., Ortega, A.E. and Gonzalez, S.A. (2018), "Small unmanned aerial vehicle (sUAV) inspections in GPS denied area beneath bridges", Proceedings of the Structures Congress 2018: Bridges, Transportation Structures, and Nonbuilding Structures, pp. 205-216.
2 Wang, R., Ma, G., Qin, Q., Shi, Q. and Huang, J. (2018), "Blind UAV images deblurring based on discriminative networks", Sensors, 18(9), 2874. https://doi.org/10.3390/s18092874   DOI
3 Xie, S., Girshick, R., Dollar, P., Tu, Z. and He, K. (2017), "Aggregated residual transformations for deep neural networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492-1500.
4 Kim, J., Oh, J. and Park, R.H. (2016), "Removing non-uniform camera shake using blind motion deblurring", Proceedings of the 2016 IEEE International Conference on Consumer Electronics (ICCE), pp. 351-352.
5 Kim, I.H., Jeon, H., Baek, S.C., Hong, W.H. and Jung, H.J. (2018), "Application of crack identification techniques for an aging concrete bridge inspection using an unmanned aerial vehicle", Sensors, 18(6), 1881. https://doi.org/10.3390/s18061881   DOI
6 Lei, J., Zhang, S., Luo, L., Xiao, J. and Wang, H. (2018), "Superresolution enhancement of UAV images based on fractional calculus and POCS", Geo-spatial Inform. Sci., 21(1), 56-66. https://doi.org/10.1080/10095020.2018.1424409   DOI
7 Liu, L., Liu, B., Huang, H. and Bovik, A.C. (2014a), "Noreference image quality assessment based on spatial and spectral entropies", Signal Process.: Image Commun., 29(8), 856-863. https://doi.org/10.1016/j.image.2014.06.006   DOI
8 Morgenthal, G. and Hallermann, N. (2014), "Quality assessment of unmanned aerial vehicle (UAV) based visual inspection of structures", Adv. Struct. Eng., 17(3), 289-302. https://doi.org/10.1260/1369-4332.17.3.289   DOI
9 Zhou, X., Mateos, J., Zhou, F., Molina, R. and Katsaggelos, A.K. (2015), "Variational Dirichlet blur kernel estimation", IEEE Transact. Image Process., 24(12), 5127-5139. https://doi.org/10.1109/TIP.2015.2478407   DOI
10 Liu, Q.F., Xiao, S.F., Huang, K.Z. and Zhong, Z. (2014b), "A SVD-based optical MIMO precoding scheme in indoor visible light communication", Int. J. Future Comput. Commun., 3(6), 421. https://doi.org/10.7763/IJFCC.2014.V3.340   DOI
11 Yoon, S., Gwon, G.H., Lee, J.H. and Jung, H.J. (2020), "Three-dimensional image coordinate-based missing region of interest area detection and damage localization for bridge visual inspection using unmanned aerial vehicles", Struct. Health Monitor., 1475921720918675. https://doi.org/10.1177/1475921720918675   DOI
12 Dong, C., Loy, C.C. and Tang, X. (2016), "Accelerating the superresolution convolutional neural network", Proceedings of European Conference on Computer Vision, pp. 391-407.
13 Burdziakowski, P. (2020), "A novel method for the deblurring of photogrammetric images using conditional generative adversarial networks", Remote Sensing, 12(16), 2586. https://doi.org/10.3390/rs12162586   DOI
14 Cho, S. and Lee, S. (2009), "Fast motion deblurring", ACM SIGGRAPH Asia 2009 papers, pp. 1-8. https://doi.org/10.1145/1661412.1618491   DOI
15 Cunningham, C.S., Ransom, D., Wilkes, J., Bishop, J. and White, B. (2015), "Mechanical Design Features of a Small Gas Turbine for Power Generation in Unmanned Aerial Vehicles", Proceedings of Turbo Expo: Power for Land, Sea, and Air, 56796, V008T23A021.
16 Duque, L., Seo, J. and Wacker, J. (2018), "Bridge deterioration quantification protocol using UAV", J. Bridge Eng., 23(10), 04018080. https://doi.org/(ASCE)BE.1943-5592.0001289   DOI
17 Hallermann, N. and Morgenthal, G. (2014), "Visual inspection strategies for large bridges using Unmanned Aerial Vehicles (UAV)", Proceedings of the 7th IABMAS, International Conference on Bridge Maintenance, Safety and Management, pp. 661-667.
18 Liu, H., Wang, W., He, Z., Tong, Q., Wang, X., Yu, W. and Lv, M. (2015), "Blind image quality evaluation metrics design for UAV photographic application", Proceedings of IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 293-297.
19 Liu, Y., Yeoh, J.K. and Chua, D.K. (2020), "Deep learning-based enhancement of motion blurred UAV concrete crack images", J. Comput. Civil Eng., 34(5), 04020028. https://doi.org/(ASCE)CP.1943-5487.0000907   DOI
20 Golestaneh, S.A. and Karam, L.J. (2017), "Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes", CVPR 2017, pp. 596-605.
21 Gao, J., Liao, W., Nuyttens, D., Lootens, P., Vangeyte, J., Pizurica, A. and Pieters, J.G. (2018), "Fusion of pixel and object-based features for weed mapping using unmanned aerial vehicle imagery", Int. J. Appl. Earth Observ. Geoinform., 67, 43-53. https://doi.org/10.1016/j.jag.2017.12.012   DOI
22 Jung, H.J., Lee, J.H., Yoon, S. and Kim, I.H. (2019), "Bridge inspection and condition assessment using unmanned aerial vehicles (UAVs): major challenges and solutions from a practical perspective", Smart Struct. Syst., Int. J., 24(5), 669-681. https://doi.org/10.12989/sss.2019.24.5.669   DOI
23 He, K., Gkioxari, G., Dollar, P. and Girshick, R. (2017), "Mask RCNN", Proceedings of the IEEE International Conference on Computer Vision, pp. 2961-2969.
24 Jeong, G.Y., Nguyen, T.N., Tran, D.K. and Hoang, T.B.H. (2020), "Applying unmanned aerial vehicle photogrammetry for measuring dimension of structural elements in traditional timber building", Measurement, 153, 107386. https://doi.org/10.1016/j.measurement.2019.107386   DOI
25 Kim, B. and Cho, S. (2019), "Image-based concrete crack assessment using mask and region-based convolutional neural network", Struct. Control Health Monitor., 26(8), e2381. https://doi.org/10.1002/stc.2381   DOI
26 Jordan, S., Moore, J., Hovet, S., Box, J., Perry, J., Kirsche, K. and Tse, Z.T.H. (2018), "State-of-the-art technologies for UAV inspections", IET Radar, Sonar & Navigation, 12(2), 151-164. https://doi.org/10.1049/iet-rsn.2017.0251   DOI
27 Myeong, W. and Myung, H. (2018), "Development of a wallclimbing drone capable of vertical soft landing using a tilt-rotor mechanism", IEEE Access, 7, 4868-4879. https://doi.org/10.1109/ACCESS.2018.2889686   DOI
28 Mittal, A., Moorthy, A.K. and Bovik, A.C. (2012a), "No-reference image quality assessment in the spatial domain", IEEE Transactions on Image Processing, 21(12), 4695-4708. https://doi.org/ 10.1109/TIP.2012.2214050   DOI
29 Mittal, A., Soundararajan, R. and Bovik, A.C. (2012b), "Making a "completely blind" image quality analyzer", IEEE Signal Processing Letters, 20(3), 209-212. https://doi.org/ 10.1109/LSP.2012.2227726   DOI
30 Morgenthal, G., Hallermann, N., Kersten, J., Taraben, J., Debus, P., Helmrich, M. and Rodehorst, V. (2019), "Framework for automated UAS-based structural condition assessment of bridges", Automat. Constr., 97, 77-95. https://doi.org/10.1016/j.autcon.2018.10.006   DOI
31 O'Connor, J., Smith, M.J. and James, M.R. (2017), "Cameras and settings for aerial surveys in the geosciences: Optimising image data", Progress Phys. Geography, 41(3), 325-344. https://doi.org/10.1177/0309133317703092   DOI
32 Sieberth, T., Wackrow, R. and Chandler, J.H. (2016), "Automatic detection of blurred images in UAV image sets", ISPRS J. Photogrammetry Remote Sens., 122, 1-16. https://doi.org/10.1016/j.isprsjprs.2016.09.010   DOI
33 Salaan, C.J.O., Okada, Y., Mizutani, S., Ishii, T., Koura, K., Ohno, K. and Tadokoro, S. (2018), "Close visual bridge inspection using a UAV with a passive rotating spherical shell", J. Field Robot., 35(6), 850-867. https://doi.org/10.1002/rob.21781   DOI
34 Seo, J., Duque, L. and Wacker, J. (2018), "Drone-enabled bridge inspection methodology and application", Automat. Constr., 94, 112-126. https://doi.org/10.1016/j.autcon.2018. 06.006   DOI
35 Sieberth, T., Wackrow, R. and Chandler, J.H. (2015), "UAV image blur - its influence and ways to correct it", The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 40(1), 33. https://doi.org/10.5194/isprsarchives-XL-1-W4-33-2015   DOI
36 Su, B., Lu, S. and Tan, C.L. (2011), "Blurred image region detection and classification", Proceedings of the 19th ACM International Conference on Multimedia, pp. 1397-1400.
37 Dorafshan, S., Maguire, M., Hoffer, N.V. and Coopmans, C. (2017), "Challenges in bridge inspection using small unmanned aerial systems: Results and lessons learned", Proceedings of the 2017 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1722-1730.