Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Assembly performance evaluation method for prefabricated steel structures using deep learning and k-nearest neighbors

  • Hyuntae Bang (Department of Autonomous Vehicle System Engineering, Chungnam National University) ;
  • Byeongjun Yu ( StradVision) ;
  • Haemin Jeon (Department of Civil and Environmental Engineering, Hanbat National University)
  • Received : 2023.03.10
  • Accepted : 2023.08.14
  • Published : 2023.08.25
⟨ Previous Next ⟩

Abstract

This study proposes an automated assembly performance evaluation method for prefabricated steel structures (PSSs) using machine learning methods. Assembly component images were segmented using a modified version of the receptive field pyramid. By factorizing channel modulation and the receptive field exploration layers of the convolution pyramid, highly accurate segmentation results were obtained. After completing segmentation, the positions of the bolt holes were calculated using various image processing techniques, such as fuzzy-based edge detection, Hough's line detection, and image perspective transformation. By calculating the distance ratio between bolt holes, the assembly performance of the PSS was estimated using the k-nearest neighbors (kNN) algorithm. The effectiveness of the proposed framework was validated using a 3D PSS printing model and a field test. The results indicated that this approach could recognize assembly components with an intersection over union (IoU) of 95% and evaluate assembly performance with an error of less than 5%.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2021R1F1A1057721). It was also supported by the "National R&D Project for Smart Construction Technology (RS-2020-KA156007)" funded by the Korea Agency for Infrastructure Technology Advancement under the Ministry of Land, Infrastructure and Transport, and managed by the Korea Expressway Corporation. The authors extend special thanks to Daewoo ST for their cooperation.

References

  1. Badrinarayanan, V., Kendall, A. and Cipolla, R. (2017), "Segnet: A deep convolutional encoder- decoder architecture for image segmentation", IEEE Trans. Pattern Anal. Mach. Intel., 39(12), 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615.
  2. Bhatia, N. (2010), Survey of Nearest Neighbor Techniques, arXiv preprint arXiv:1007.0085.
  3. Boafo, F.E., Kim, J.H. and Kim, J.T. (2016), "Performance of modular prefabricated architecture: Case study-based review and future pathways", Sustain., 8(6), 558. https://doi.org/10.3390/su8060558.
  4. Chen, L.C., Papandreou, G., Schroff, F. and Adam, H. (2017), Rethinking Atrous Convolution for Semantic Image Segmentation, arXiv preprint arXiv:1706.05587.
  5. Choi, S., Myeong, W., Jeong, Y. and Myung, H. (2017), "Vision-based hybrid 6-DOF displacement estimation for precast concrete member assembly", Smart Struct. Syst., 20(4), 397-413. https://doi.org/10.12989/sss.2017.20.4.397.
  6. Construction Workers Mutual Aid Association (2021), Trends in Construction Experienced Workers and Construction Skilled Workers, https://cwma.bigzine.kr/. (in Korean)
  7. Cover, T. and Hart, P. (1967), "Nearest neighbor pattern classification", IEEE Trans. Inform. Theory, 13(1), 21-27. https://doi.org/10.1109/TIT.1967.1053964.
  8. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H. and Wei, Y. (2017), "Deformable convolutional networks", Proceedings of the IEEE International Conference on Computer Vision, 764-773.
  9. Deng, E.F., Zong, L., Ding, Y., Zhang, Z., Zhang, J.F., Shi, F.W., Cai, L.M. and Gao, S.C. (2020), "Seismic performance of mid-to-high rise modular steel construction-A critical review", Thin Wall. Struct., 155, 106924. https://doi.org/10.1016/j.tws.2020.106924.
  10. Geron, A. (2019), Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, O'Reilly Media, Inc..
  11. Hinton, G. and Sejnowski, T.J. (1999), Unsupervised Learning: Foundations of Neural Computation, MIT Press.
  12. Jegou, S., Drozdzal, M., Vazquez, D., Romero, A. and Bengio, Y. (2017), "The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 11-19.
  13. Jeon, H., Kim, Y., Lee, D. and Myung, H. (2014), "Vision-based remote 6-DOF structural displacement monitoring system using a unique marker", Smart Struct. Syst., 13(6), 927-942. https://doi.org/10.12989/sss.2014.13.6.927.
  14. Kim, M.K., Sohn, H. and Chang, C.C. (2014), "Automated dimensional quality assessment of precast concrete panels using terrestrial laser scanning", Auto. Constr., 45, 163-177. https://doi.org/10.1016/j.autcon.2014.05.015.
  15. Kim, M.K., Wang, Q., Park, J.W., Cheng, J.C., Sohn,H. and Chang, C.C. (2016), "Automated dimensional quality assurance of full-scale precast concrete elements using laser scanning and BIM", Auto. Constr., 72, 102-114. https://doi.org/10.1016/j.autcon.2016.08.035.
  16. Koem, C., Shim, C.S. and Park, S.J. (2016), "Seismic performance of prefabricated bridge columns with combination of continuous mild reinforcements and partially unbonded tendons", Smart Struct. Syst., 17(4), 541-557. https://doi.org/10.12989/sss.2016.17.4.541.
  17. Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012), "Imagenet classification with deep convolutional neural networks", Adv. Neur. Inform. Proc. Syst., 25, 1.
  18. Liu, S. and Huang, D. (2018), "Receptive field block net for accurate and fast object detection", Proceedings of the European Conference on Computer Vision (ECCV), 385-400.
  19. Liu, S., De Mello, S., Gu, J., Zhong, G., Yang, M.H. and Kautz, J. (2017), "Learning affinity via spatial propagation networks", Advances in Neural Information Processing Systems, 30.
  20. Long, J., Shelhamer, E. and Darrell, T. (2015), "Fully convolutional networks for semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3431-3440.
  21. McLachlan, G.J. (2005), Discriminant Analysis and Statistical Pattern Recognition, John Wiley & Sons.
  22. Mehta, S., Rastegari, M., Caspi, A., Shapiro, L. and Hajishirzi, H. (2018), "Espnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation", Proceedings of the European Conference on Computer Vision (ECCV), 552- 568.
  23. Ministry of Land, Infrastructure, and Transport (2013), Road Bridge Standard Specification, https://www.codil.or.kr/.
  24. Ministry of Land, Infrastructure, and Transport (2021), Guidelines for Activation of Smart Construction Technology (Notice No. 2021-1283), https://www.law.go.kr/.
  25. Mitchell, T.M. (2007), Machine Learning, Volume 1, McGraw-Hill, New York.
  26. Myles, A.J., Feudale, R.N., Liu, Y., Woody, N.A. and Brown, S.D. (2004), "An introduction to decision tree modeling", J. Chemometr.: J. Chemometr. Soc., 18(6), 275-285. https://doi.org/10.1002/cem.873.
  27. Nguyen, D.C., Park, S.J. and Shim, C.S. (2022), "Digital engineering models for prefabricated bridge piers", Smart Struct. Syst., 30(1), 35-47. https://doi.org/10.12989/sss.2022.30.1.035.
  28. Noble, W.S. (2006), "What is a support vector machine?", Nat. Biotechnol., 24(12), 1565-1567. https://doi.org/10.1038/nbt1206-1565.
  29. Noh, H., Hong, S. and Han, B. (2015), "Learning deconvolution network for semantic segmentation", Proceedings of the IEEE International Conference on Computer Vision, 1520-1528.
  30. Park, H.S., Lee, H., Adeli, H. and Lee, I. (2007), "A new approach for health monitoring of structures: terrestrial laser scanning", Comput.-Aid. Civil Infrastr. Eng., 22(1), 19-30. https://doi.org/10.1111/j.1467-8667.2006.00466.x.
  31. Rahman, I.A., Memon, A.H. and Karim, A.T.A. (2013), "Relationship between factors of construction resources affecting project cost", Modern Appl. Sci., 7(1), 67-75.
  32. Ronneberger, O., Fischer, P. and Brox, T. (2015), "U-net: Convolutional networks for biomedical image segmentation", Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October.
  33. Simonyan, K. and Zisserman, A. (2014), Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv preprint arXiv:1409.1556.
  34. Stefanic, M. and Stankovski, V. (2018), "A review of technologies and applications for smart construction", Proc. Inst. Civil Eng.-Civil Eng., 172, 83-87. https://doi.org/10.1680/jcien.17.00050.
  35. Suthaharan, S. (2016), "Machine learning models and algorithms for big data classification", Integr. Ser. Inf. Syst., 36, 1-12. https://doi.org/10.1007/978-1-4899-7641-3.
  36. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A. (2015), "Going deeper with convolutions", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1- 9.
  37. Versaci, M. and Morabito, F.C. (2021), "Image edge detection: A new approach based on fuzzy entropy and fuzzy divergence", Int. J. Fuzzy Syst., 23(4), 918-936. https://doi.org/10.1007/s40815-020-01030-5.
  38. Wang, H., Su, D., Liu, C., Jin, L., Sun, X. and Peng, X. (2019), "Deformable non-local network for video super-resolution", IEEE Access, 7, 177734-177744. https://doi.org/10.1109/ACCESS.2019.2958030.
  39. Wasim, M., Vaz Serra, P. and Ngo, T.D. (2020), "Design for manufacturing and assembly for sustainable, quick and cost-effective prefabricated construction-a review", Int. J. Constr. Manage., 1-9. https://doi.org/10.1080/15623599.2020.1837720.
  40. Xu, Z., Kang, R. and Lu, R. (2020), "3D reconstruction and measurement of surface defects in prefabricated elements using point clouds", J. Comput. Civil Eng., 34(5), 04020033. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000920.
  41. Yu, B., Jeon, H., Bang, H., Yi, S.S. and Min, J. (2022), "Fender segmentation in unmanned aerial vehicle images based on densely connected receptive field block", Int. J. Nav. Arch. Ocean Eng., 14, 100472. https://doi.org/10.1016/j.ijnaoe.2022.100472.
  42. Yu, F. and Koltun, V. (2015), Multi-scale Context Aggregation by Dilated Convolutions, arXiv preprint arXiv:1511.07122.
  43. Yu, F., Koltun, V. and Funkhouser, T. (2017), "Dilated residual networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 472-480.
  44. Zonta, D., Pozzi, M. and Bursi, O.S. (2007), "Performance evaluation of smart prefabricated concrete elements", Smart Struct. Syst., 3(4), 475-494. https://doi.org/10.12989/sss.2007.3.4.475.