KSCE Journal of Civil and Environmental Engineering Research (대한토목학회논문집)

Korean Society of Civil Engeneers (대한토목학회)
권호 표지
DOI QR코드

DOI QR Code

Solitary Work Detection of Heavy Equipment Using Computer Vision

컴퓨터비전을 활용한 건설현장 중장비의 단독작업 자동 인식 모델 개발

  • 정인수 (서울대학교 건설환경공학부) ;
  • 김진우 (서울대학교 건설환경종합연구소) ;
  • 지석호 (서울대학교 건설환경공학부) ;
  • 노명일 (서울대학교 조선해양공학과) ;
  • Received : 2020.08.25
  • Accepted : 2021.02.01
  • Published : 2021.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

Construction sites are complex and dangerous because heavy equipment and workers perform various operations simultaneously within limited working areas. Solitary works of heavy equipment in complex job sites can cause fatal accidents, and thus they should interact with spotters and obtain information about surrounding environments during operations. Recently, many computer vision technologies have been developed to automatically monitor construction equipment and detect their interactions with other resources. However, previous methods did not take into account the interactions between equipment and spotters, which is crucial for identifying solitary works of heavy equipment. To address the drawback, this research develops a computer vision-based solitary work detection model that considers interactive operations between heavy equipment and spotters. To validate the proposed model, the research team performed experiments using image data collected from actual construction sites. The results showed that the model was able to detect workers and equipment with 83.4 % accuracy, classify workers and spotters with 84.2 % accuracy, and analyze the equipment-to-spotter interactions with 95.1 % accuracy. The findings of this study can be used to automate manual operation monitoring of heavy equipment and reduce the time and costs required for on-site safety management.

건설현장에서는 수많은 중장비와 작업자가 다양한 작업을 동시다발적으로 수행하기 때문에 복잡하고 위험한 상황이 자주 발생한다. 복잡한 현장에서 중장비가 단독으로 작업할 경우 운전자의 시야제한, 판단오류 등으로 인해 안전사고가 발생할 수 있으며, 이에 따라 중장비는 신호수와의 상호작용을 통해 주변 상황에 대한 정보를 수집하면서 작업을 수행해야 한다. 중장비를 자동으로 모니터링하고 위험상황을 탐지하기 위해 많은 컴퓨터비전 기술들이 개발되었지만, 기존의 방법들은 중장비 단독작업 인식에 필요한 중장비와 신호수 간 상호작용을 고려하지 않았다는 한계가 있다. 이러한 한계를 보완하기 위해 본 연구는 중장비-신호수 간의 상호작용을 고려한 컴퓨터비전 기반 중장비의 단독작업 자동 인식 모델을 제안함을 목표로 한다. 개발된 모델을 검증하기 위해 실제 건설현장으로부터 영상 데이터를 수집하여 실험을 수행하였다. 그 결과, 학습된 모델은 중장비와 사람을 83.4 %의 정확도로 인식하였고, 일반 작업자와 신호수를 84.2 %의 정확도로 분류하였으며, 중장비-신호수 간 상호작용 또한 95.1 %의 높은 정확도로 분석하였다. 본 연구결과는 건설현장에서 위험한 상황을 초래할 수 있는 중장비의 단독작업을 사전에 탐지하여 안전사고를 예방할 수 있다.

Keywords

Acknowledgement

본 연구는 국토교통부/국토교통과학기술진흥원의 지원으로 수행되었음(과제번호 20SMIP-A158708-01). 이 논문은 2018년도 서울대학교 융·복합 연구과제 지원사업의 지원을 받아 수행된 연구임.

References

  1. Arabi, S., Haghighat, A. and Sharma, A. (2019). A deep learning based solution for construction equipment detection: from development to deployment. arXiv preprint arXiv:1904.09021.
  2. Azar, E. R. and McCabe, B. (2012a). "Part based model and spatial-temporal reasoning to recognize hydraulic excavators in construction images and videos." Automation in Construction, Vol. 24, pp. 194-202. https://doi.org/10.1016/j.autcon.2012.03.003
  3. Azar, E. R. and McCabe, B. (2012b). "Automated visual recognition of dump trucks in construction videos." Journal of Computing in Civil Engineering, Vol. 26, No. 6, pp. 769-781. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000179
  4. Cai, J., Zhang, Y. and Cai, H. (2019). "Two-step long short-term memory method for identifying construction activities through positional and attentional cues." Automation in Construction, Vol. 106, pp. 102886. https://doi.org/10.1016/j.autcon.2019.102886
  5. Chi, S. H. and Caldas, C. H. (2011). "Automated object identification using optical video cameras on construction sites." Computer-Aided Civil and Infrastructure Engineering, Vol. 26, No. 5, pp. 368-380. https://doi.org/10.1111/j.1467-8667.2010.00690.x
  6. Chi, S. H. and Caldas, C. H. (2012). "Image-based safety assessment: Automated spatial safety risk identification of earthmoving and surface mining activities." Journal of Construction Engineering and Management, Vol. 138, No. 3, pp. 341-351. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000438
  7. Chi, S. H., Caldas, C. H. and Kim, D. Y. (2009). "A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques." Computer-Aided Civil and Infrastructure Engineering, Vol. 24, No. 3, pp. 199-211. https://doi.org/10.1111/j.1467-8667.2008.00580.x
  8. Construction Safety Management Information System (COSMIS) (2015). Construction accident DB: 2011~2015 (in Korean), Available at: https://www.csi.go.kr/ (Accessed: November 11, 2020).
  9. DarkLabel 1.3 (2018). Image labeling and annotation tool, Available at: https://darkpgmr.tistory.com/16 (Accessed: July 20, 2020).
  10. Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T. M. and An, W. (2018). "Detecting non-hardhat-use by a deep learning method from far-field surveillance videos." Automation in Construction, Vol. 85, pp. 1-9. https://doi.org/10.1016/j.autcon.2017.09.018
  11. Kim, H. J., Kim, K. N. and Kim, H. K. (2016). "Vision-based object-centric safety assessment using fuzzy inference: Monitoring struck-by accidents with moving objects." Journal of Computing in Civil Engineering, Vol. 30, No. 4, pp. 1-13.
  12. Kim, J. W. and Chi, S. H. (2017). "Adaptive detector and tracker on construction sites using functional integration and online learning." Journal of Computing in Civil Engineering, Vol. 31, No. 5, pp. 1-13.
  13. Kim, J. W., Chi, S. H. and Seo, J. W. (2018). "Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks." Automation in Construction, Vol. 87, pp. 297-308. https://doi.org/10.1016/j.autcon.2017.12.016
  14. Korea Construction Technology Promotion Act (2016). Enforcement decree article 98 and 99, statutes of the Republic of Korea (in Korean).
  15. Korea Occupational Safety & Health Agency (KOSHA) (2011). Risk assessment on construction industry by work type (in Korean).
  16. Li, H., Chan, G., Wong, J. K. W. and Skitmore, M. (2016). "Real-time locating systems applications in construction." Automation in Construction, Vol. 63, pp. 37-47. https://doi.org/10.1016/j.autcon.2015.12.001
  17. MATLAB (2019). A proprietary multi-paradigm programming language and numerical computing environment: developed by MathWorks, Available at: https://kr.mathworks.com/ (Accessed: November 11, 2020).
  18. Memarzadeh, M., Golparvar-Fard, M. and Niebles, J. C. (2013). "Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors." Automation in Construction, Vol. 32, pp. 24-37. https://doi.org/10.1016/j.autcon.2012.12.002
  19. Park, M. W. and Brilakis, I. (2012). "Construction worker detection in video frames for initializing vision trackers." Automation in Construction, Vol. 28, pp. 15-25. https://doi.org/10.1016/j.autcon.2012.06.001
  20. Seoul Metropolitan Government Construction Specification (SMGCS) (2018). Safety and health management (in Korean).
  21. Yu, Y., Guo, H., Ding, Q., Li, H. and Skitmore, M. (2017). "An experimental study of real-time identification of construction workers' unsafe behaviors." Automation in Construction, Vol. 82, pp. 193-206. https://doi.org/10.1016/j.autcon.2017.05.002
  22. Zhu, Z., Ren, X. and Chen, Z. (2016). "Visual tracking of construction jobsite workforce and equipment with particle filtering." Journal of Computing in Civil Engineering, Vol. 30, No. 6, pp. 1-15.