Browse > Article
http://dx.doi.org/10.12989/sss.2021.27.5.783

Detection and quantification of bolt loosening using RGB-D camera and Mask R-CNN  

Chung, Junyeon (Department of Civil and Environmental Engineering, Korea Advanced Institute for Science and Technology)
Sohn, Hoon (Department of Civil and Environmental Engineering, Korea Advanced Institute for Science and Technology)
Publication Information
Smart Structures and Systems / v.27, no.5, 2021 , pp. 783-793 More about this Journal
Abstract
Bolt loosening is one of the most common types of damage for bolt-connected plates. Existing vision techniques detect bolt loosening based on the measurement of bolt rotation or the exposure of bolt threads. However, these techniques examine bolt tightness only in a qualitative manner, or require a reference measurement at the initially tightened state of the bolt for quantitative estimation. In this study, the exposed shank length of a bolt is quantitatively measured using an RGB-depth camera and a mask-region-based convolutional neural network but without requiring any measurement from the initial state of the bolt. The performance of the proposed technique is validated by conducting lab-scale experiments, in which the angle and distance of the camera are varied with respect to a target inspection area. The proposed technique successfully detects bolt loosening at exposed shank length over 3 mm with a resolution of 1 mm and 97% accuracy at different camera angles (40°-90°) and distances (up to 65 cm).
Keywords
bolt-loosening detection; bolt-loosening quantification; RGB-depth camera; Mask R-CNN; deep learning;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Lin, T.-Y., Maire, M., Belongie, S., Hays, J, Perona, P., Ramanan, D., Dollar, P. and Zitnick C.L. (2014), "Microsoft COCO: common objects in context", Proceedings of European Conference on Computer Vision, Zurich, Switzerland, September.
2 Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S. and Cheng-Yang, F. (2016), "SSD: single shot multibox detector", Proceedings of European Conference on Computer Vision, Amsterdam, The Netherlands, October.
3 Park, J.-H., Huynh, T.-C., Choi, S.-H. and Kim, J.-T. (2015), "Vision-based technique for bolt-loosening detection in wind turbine tower", Wind Struct., Int. J., 21(6), 709-726. https://doi.org/10.12989/was.2015.21.6.709   DOI
4 Ramana, L., Choi, W. and Cha, Y.-J. (2019), "Fully automated vision-based loosened bolt detection using the Viola-Jones algorithm", Struct. Health Monit., 18(2), 422-434. https://doi.org/10.1177/1475921718757459   DOI
5 Redmon, J. and Farhadi, A. (2018), "YOLOv3: An incremental improvement", arXiv Prepr. arXiv1804.02767.
6 Ren, S., He, K., Girshick, R. and Sun, J. (2015), "Faster R-CNN: towards real-time object detection with region proposal networks", Neural Inform. Process. Syst., Montreal, Canada, December.
7 Simonyan, K. and Zisserman, A. (2014), "Very deep convolutional networks for large-scale image recognition", arXiv Prepr. arXiv1409.1556.
8 Torrey, L. and Shavlik, J. (2010), "Transfer learning", in Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, IGI Global, PA, USA.
9 Van Dyk, D.A. and Meng, X.-L. (2001), "The art of data augmentation", J. Comput. Graph. Stat., 10(1), 1-50. https://doi.org/10.1198/10618600152418584   DOI
10 Suda, M., Hasuo, Y., Kanaya, A., Ogura, Y. Takishita, T. and Suzuki, Y. (1992), "Development of ultrasonic axial bolting force inspection system for turbine bolts in thermal power plants", JSME Int. J. Ser. 1, Solid Mech. Strength Mater., 35(2), 216-219. https://doi.org/10.1299/jsmea1988.35.2_216   DOI
11 Huynh, T.-C., Park, J.-H., Jung, H.-J. and Kim, J.-T. (2019), "Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing", Autom. Constr., 105, 102844. https://doi.org/10.1016/j.autcon.2019.102844   DOI
12 Dubois, E. and Sabri, S. (1984), "Noise reduction in image sequences using motion-compensated temporal filtering", IEEE Trans. Commun., 32(7), 826-831. https://doi.org/10.1109/TCOM.1984.1096143   DOI
13 Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B. and Belongie, S. (2017), "Feature pyramid networks for object detection", Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, July.
14 Huynh, T. and Kim, J. (2018), "RBFN-based temperature compensation method for impedance monitoring in prestressed tendon anchorage", Struct. Control Health Monit., 25(6), e2173. https://doi.org/10.1002/stc.2173   DOI
15 Huynh, T.-C., Dang, N.-L. and Kim, J.-T. (2018), "Preload monitoring in bolted connection using piezoelectric-based smart interface", Sensors, 18(9), 2766. https://doi.org/10.3390/s18092766   DOI
16 Korea Expressway Corporation (2013), "Improvement of bridge inspection system by the damage analysis", Korea Expressway Corporation.
17 Hartley, R. and Zisserman, A. (2003), Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge, United Kingdom.
18 Benkhoui, Y., El Korchi, T. and Reinhold, L. (2019), "UAS-based crack detection using stereo cameras: a comparative study", Proceedings of International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, June.
19 Cha, Y.-J., You, K. and Choi, W. (2016), "Vision-based detection of loosened bolts using the Hough transform and support vector machines", Autom. Constr., 71(2), 181-188. https://doi.org/10.1016/j.autcon.2016.06.008   DOI
20 Fischler, M.A. and Bolles, R.C. (1981), "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography", Commun. ACM, 24(6), 381-395. https://doi.org/10.1145/358669.358692   DOI
21 He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, June.
22 Zhao, X., Zhang, Y. and Wang, N. (2019), "Bolt loosening angle detection technology using deep learning", Struct. Control Health Monit., 26(1), e2292. https://doi.org/10.1002/stc.2292   DOI
23 He, K., Gkioxari, G., Dollar, P. and Girshick, R. (2017), "Mask R-CNN", IEEE International Conference on Computer Vision, Venice, Italy, October.
24 Grunnet-Jepsen, A., Sweetser, J.N., Winer, P., Takagi, A. and Woodfill, J. (2018), "Projectors for Intel® RealSenseTM Depth Cameras D4xx", Intel.
25 Huynh, T.-C. and Kim, J.-T. (2017), "Quantification of temperature effect on impedance monitoring via PZT interface for prestressed tendon anchorage", Smart Mater. Struct., 26(12), 125004. https://doi.org/10.1088/1361-665X/aa931b   DOI
26 Wang, T., Song, G., Liu, S., Li, Y. and Xiao, H. (2013a), "Review of bolted connection monitoring", Int. J. Distrib. Sens. Netw., 9(12), 871213. https://doi.org/10.1155/2013/871213   DOI
27 Wang, T., Song, G., Wang, Z. and Li, Y. (2013b), "Proof-of-concept study of monitoring bolt connection status using a piezoelectric based active sensing method", Smart Mater. Struct., 22(8), 87001. https://doi.org/10.1088/0964-1726/22/8/087001   DOI
28 Zhang, Y., Sun, X., Loh, K.J., Su, W., Xue, Z. and Zhao, X. (2019), "Autonomous bolt loosening detection using deep learning", Struct. Health Monit., 19(1), 105-122. https://doi.org/10.1177/1475921719837509   DOI