International conference on construction engineering and project management (국제학술발표논문집)

  • 2024.07a
  • /
  • Pages.791-798
  • /
  • 2024
  • /
  • 2508-9048(eISSN)
Korea Institute of Construction Engineering and Management (한국건설관리학회)
권호 표지
DOI QR코드

DOI QR Code

Computer Vision-based Construction Hazard Detection via Data Augmentation Approach using Generative-AI

  • WooWon Jo (Department of Architectural Engineering, Pusan National University) ;
  • YeJun Lee (Department of Architectural Engineering, Pusan National University) ;
  • Daegyo Jung (Department of Architectural Engineering, Pusan National University) ;
  • HyunJung Park (Department of Architecture, Silla University) ;
  • JungHo Jeon (Department of Architectural Engineering, Pusan National University)
  • Published : 2024.07.29
Download PDF
⟨ Previous Next ⟩

Abstract

Construction industry records poor safety records annually due to a large number of injuries and accidents on construction jobsite. In order to improve existing safety performance, object detection approaches have been extensively studied using vision-sensing techniques and deep learning algorithms. Unfortunately, an insufficient number of datasets (e.g., images) and challenges that reside in manually collecting quality datasets constitute a significant hurdle in fully deploying object recognition approaches in real construction sites. Although advanced technologies (e.g., virtual reality) have attempted to address such challenges, they have achieved limited success because they still rely on labor-intensive work. A promising alternative is to adopt generative AI-based data augmentation methods attributed to their efficiency in creating realistic visual datasets and proven performance. However, there remain critical knowledge gaps on how such alternatives can be effectively employed by safety managers on real construction sites in terms of practicability and applications. In this context, this study establishes a framework that can identify effective strategies for improving object detection performance (e.g., accuracy) using generative AI technologies. The outcome of this study will contribute to providing guidelines and best practices for practitioners as well as researchers by exploring different generative AI-driven augmentation approaches and comparing the corresponding results in a quantitative manner.

Keywords

References

  1. J. Park, K. Kim, Y.K. Cho, Framework of Automated Construction-Safety Monitoring Using Cloud-Enabled BIM and BLE Mobile Tracking Sensors, J. Constr. Eng. Manage. 143 (2017) 05016019. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001223. 
  2. P. Mitropoulos, T.S. Abdelhamid, G.A. Howell, Systems Model of Construction Accident Causation, J. Constr. Eng. Manage. 131 (2005) 816-825. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:7(816). 
  3. S. Taneja, B. Akinci, J.H. Garrett, L. Soibelman, E. Ergen, A. Pradhan, P. Tang, M. Berges, G. Atasoy, X. Liu, S.M. Shahandashti, E.B. Anil, Sensing and Field Data Capture for Construction and Facility Operations, J. Constr. Eng. Manage. 137 (2011) 870-881. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000332. 
  4. J. Seo, S. Han, S. Lee, H. Kim, Computer vision techniques for construction safety and health monitoring, Advanced Engineering Informatics 29 (2015) 239-251. https://doi.org/10.1016/j.aei.2015.02.001. 
  5. Y. Liu, P. Sun, N. Wergeles, Y. Shang, A survey and performance evaluation of deep learning methods for small object detection, Expert Systems with Applications 172 (2021) 114602. https://doi.org/10.1016/j.eswa.2021.114602. 
  6. F. Fui-Hoon Nah, R. Zheng, J. Cai, K. Siau, L. Chen, Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration, Journal of Information Technology Case and Application Research 25 (2023) 277-304. https://doi.org/10.1080/15228053.2023.2233814. 
  7. Y. Ge, J. Xu, B.N. Zhao, N. Joshi, L. Itti, V. Vineet, Beyond Generation: Harnessing Text to Image Models for Object Detection and Segmentation, (2023). https://doi.org/10.48550/ARXIV.2309.05956. 
  8. A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis, Deep Learning for Computer Vision: A Brief Review, Computational Intelligence and Neuroscience 2018 (2018) 1-13. https://doi.org/10.1155/2018/7068349. 
  9. B. Yang, B. Zhang, Q. Zhang, Z. Wang, M. Dong, T. Fang, Automatic detection of falling hazard from surveillance videos based on computer vision and building information modeling, Structure and Infrastructure Engineering 18 (2022) 1049-1063. https://doi.org/10.1080/15732479.2022.2039217. 
  10. B.E. Mneymneh, M. Abbas, H. Khoury, Automated Hardhat Detection for Construction Safety Applications, Procedia Engineering 196 (2017) 895-902. https://doi.org/10.1016/j.proeng.2017.08.022. 
  11. T. Kong, W. Fang, P.E.D. Love, H. Luo, S. Xu, H. Li, Computer vision and long short-term memory: Learning to predict unsafe behaviour in construction, Advanced Engineering Informatics 50 (2021) 101400. https://doi.org/10.1016/j.aei.2021.101400. 
  12. D.A. Van Dyk, X.-L. Meng, The Art of Data Augmentation, Journal of Computational and Graphical Statistics 10 (2001) 1-50. https://doi.org/10.1198/10618600152418584. 
  13. A. Mumuni, F. Mumuni, Data augmentation: A comprehensive survey of modern approaches, Array 16 (2022) 100258. https://doi.org/10.1016/j.array.2022.100258. 
  14. S. Bang, F. Baek, S. Park, W. Kim, H. Kim, Image augmentation to improve construction resource detection using generative adversarial networks, cut-and-paste, and image transformation techniques, Automation in Construction 115 (2020) 103198. https://doi.org/10.1016/j.autcon.2020.103198. 
  15. K.M. Rashid, J. Louis, Times-series data augmentation and deep learning for construction equipment activity recognition, Advanced Engineering Informatics 42 (2019) 100944. https://doi.org/10.1016/j.aei.2019.100944. 
  16. S. Yang, W. Xiao, M. Zhang, S. Guo, J. Zhao, F. Shen, Image Data Augmentation for Deep Learning: A Survey, (2022). https://doi.org/10.48550/ARXIV.2204.08610. 
  17. A. Biswas, N. Md Abdullah Al, A. Imran, A.T. Sejuty, F. Fairooz, S. Puppala, S. Talukder, Generative Adversarial Networks for Data Augmentation, in: B. Zheng, S. Andrei, M.K. Sarker, K.D. Gupta (Eds.), Data Driven Approaches on Medical Imaging, Springer Nature Switzerland, Cham, 2023: pp. 159-177. https://doi.org/10.1007/978-3-031-47772-0_8. 
  18. H. Ohno, Auto-encoder-based generative models for data augmentation on regression problems, Soft Comput 24 (2020) 7999-8009. https://doi.org/10.1007/s00500-019-04094-0. 
  19. D. Bai̇Doo-Anu, L. Owusu Ansah, Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning, Journal of AI 7 (2023) 52-62. https://doi.org/10.61969/jai.1337500. 
  20. Z.M. Albaghajati, D.M. Bettaieb, R.B. Malek, Exploring text-to-image application in architectural design: insights and implications, Archit. Struct. Constr. 3 (2023) 475-497. https://doi.org/10.1007/s44150-023-00103-x. 
  21. C. Cao, Z. Ding, G.-G. Lee, J. Jiao, J. Lin, X. Zhai, Elucidating STEM Concepts through Generative AI: A Multi-modal Exploration of Analogical Reasoning, (2023). https://doi.org/10.48550/ARXIV.2308.10454. 
  22. L. Reynolds, K. McDonell, Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm, in: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, ACM, Yokohama Japan, 2021: pp. 1-7. https://doi.org/10.1145/3411763.3451760. 
  23. A. Borji, Generated Faces in the Wild: Quantitative Comparison of Stable Diffusion, Midjourney and DALL-E 2, (2022). https://doi.org/10.48550/ARXIV.2210.00586. 
  24. J. Terven, D. Cordova-Esparza, A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS, (2023). https://doi.org/10.48550/ARXIV.2304.00501. 
  25. T. Chen, Q. Zeng, Research on Bubble Detection Based on Improved YOLOv8n, IEEE Access 12 (2024) 9659-9668. https://doi.org/10.1109/ACCESS.2024.3353196. 
  26. Y. Huang, W. Tan, L. Li, L. Wu, WFRE-YOLOv8s: A New Type of Defect Detector for Steel Surfaces, Coatings 13 (2023) 2011. https://doi.org/10.3390/coatings13122011.