DOI QR코드

DOI QR Code

Automated Vision-based Construction Object Detection Using Active Learning

액티브 러닝을 활용한 영상기반 건설현장 물체 자동 인식 프레임워크

  • 김진우 (서울대학교 건설환경종합연구소) ;
  • 지석호 (서울대학교 건설환경공학부) ;
  • 서준오 (홍콩이공대학)
  • Received : 2019.08.07
  • Accepted : 2019.08.28
  • Published : 2019.10.01

Abstract

Over the last decade, many researchers have investigated a number of vision-based construction object detection algorithms for the purpose of construction site monitoring. However, previous methods require the ground truth labeling, which is a process of manually marking types and locations of target objects from training image data, and thus a large amount of time and effort is being wasted. To address this drawback, this paper proposes a vision-based construction object detection framework that employs an active learning technique while reducing manual labeling efforts. For the validation, the research team performed experiments using an open construction benchmark dataset. The results showed that the method was able to successfully detect construction objects that have various visual characteristics, and also indicated that it is possible to develop the high performance of an object detection model using smaller amount of training data and less iterative training steps compared to the previous approaches. The findings of this study can be used to reduce the manual labeling processes and minimize the time and costs required to build a training database.

최근 많은 연구자들이 대규모 현장에 투입된 건설자원의 유형과 위치를 자동 파악하는 영상분석기술을 활발히 개발하고 있다. 하지만 기존의 방법들은 인식하고자 하는 건설 물체(작업자, 중장비, 자재 등)를 학습용 이미지 데이터에 표시하는 Labeling 작업을 요구하고 이에 불필요한 시간과 노력이 낭비된다는 한계가 있다. 이러한 한계를 보완하기 위해서 본 연구는 액티브 러닝을 활용한 영상기반 건설현장 물체 자동 인식 프레임 워크를 제안함을 목표로 한다. 개발 프레임워크 검증을 목적으로 건설분야 Benchmark 데이터셋을 이용하여 실제 실험을 진행하였다. 그 결과, 액티브 러닝을 통해 학습한 모델은 다양한 특성을 지닌 건설물체를 성공적으로 인식할 수 있었고, 기존의 학습 DB 구축 방식과 비교할 때 더 적은 데이터 수와 반복학습 횟수로도 높은 성능을 가지는 영상분석모델을 개발할 수 있었다. 결과적으로 기존에 요구되던 학습 DB 구축을 위한 Labeling 작업을 줄일 뿐만 아니라 총 시간과 비용을 최소화할 수 있다.

Keywords

References

  1. Azar, E. R. and McCabe, B. (2012a). "Automated visual recognition of dump trucks in construction videos." Journal of Computing in Civil Engineering, Vol. 26, No. 6, pp. 769-781. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000179
  2. Azar, E. R. and McCabe, B. (2012b). "Part based model and spatial-temporal reasoning to recognize hydraulic excavators in construction images and videos." Automation in Construction, Vol. 24, pp. 194-202. https://doi.org/10.1016/j.autcon.2012.03.003
  3. Azar, E. R., Dickinson, S. and McCabe, B. (2013). "Server-Customer interaction tracker: computer vision-based system to estimate dirt-loading cycles." Journal of Construction Engineering and Management, Vol. 139, No. 7, pp. 785-794. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000652
  4. Cai, J., Zhang, Y. and Cai, H. (2019). "Two-step long short-term memory method for identifying construction activities through positional and attentional cues." Automation in Construction, Vol. 106, p. 102886. https://doi.org/10.1016/j.autcon.2019.102886
  5. Chi, S. and Caldas, C. H. (2011). "Automated object identification using optical video cameras on construction sites." Computer-Aided Civil and Infrastructure Engineering, Vol. 26, No. 5, pp. 368-380. https://doi.org/10.1111/j.1467-8667.2010.00690.x
  6. Chi, S. and Caldas, C. H. (2012). "Image-based safety assessment: automated spatial safety risk identification of earthmoving and surface mining activities." Journal of Construction Engineering and Management, Vol. 138, No. 3, pp. 341-351. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000438
  7. Chi, S., Caldas, C. H. and Kim, D. Y. (2009). "A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques." Computer-Aided Civil and Infrastructure Engineering, Vol. 24, No. 3, pp. 199-211. https://doi.org/10.1111/j.1467-8667.2008.00580.x
  8. Chung, S. (2018). Bridge damage factor recognition from inspection reports using active recurrent neural network, Master Thesis, Seoul National University.
  9. Fang, W., Ding, L., Zhong, B., Love, P. E. D. and Luo, H. (2018). "Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach." Advanced Engineering Informatics, Vol. 37, pp. 139-149. https://doi.org/10.1016/j.aei.2018.05.003
  10. Golparvar-Fard, M., Heydarian, A. and Niebles, J. C. (2013). "Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers." Advanced Engineering Informatics, Vol. 27, No. 4, pp. 652-663. https://doi.org/10.1016/j.aei.2013.09.001
  11. Gong, J. and Caldas, C. H. (2010). "Computer vision-based video interpretation model for automated productivity analysis of construction operations." Journal of Computing in Civil Engineering, Vol. 24, No. 3, pp. 252-263. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000027
  12. Han, S. U., Achar, M., Lee, S. H. and Pena-Mora, F. (2013). "Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring." Visualization in Engineering, Vol. 1, No. 1, pp. 1-13. https://doi.org/10.1186/2213-7459-1-1
  13. Han, S. U., Lee, S. H. and Pena-mora, F. (2014). "Vision-based detection of unsafe actions of a construction worker: case study of ladder climbing." Journal of Computing in Civil Engineering, Vol. 27, No. 6, pp. 635-644. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000279
  14. Kim, H. J., Kim, H. K., Hong, Y. W. and Byun, H. R. (2018b). "Detecting construction equipment using a region-based fully convolutional network and transfer learning." Journal of Computing in Civil Engineering, Vol. 32, No. 2, p. 04017082. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000731
  15. Kim, H. J., Kim, K. N. and Kim, H. K. (2016). "Vision-based object-centric safety assessment using fuzzy inference: monitoring struck-by accidents with moving objects." Journal of Computing in Civil Engineering, Vol. 30, No. 4, p. 04015075. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000562
  16. Kim, J. W. and Chi, S. H. (2017). "Adaptive detector and tracker on constrution sites using functional integration and online learning." Journal of Computing in Civil Engineering, Vol. 31, No. 5, p. 04017026. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000677
  17. Kim, J. W. and Chi, S. H. (2019). "Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles." Automation in Construction, Vol. 104, pp. 255-264. https://doi.org/10.1016/j.autcon.2019.03.025
  18. Kim, J. W., Chi, S. H. and Seo, J. W. (2018a). "Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks." Automation in Construction, Vol. 87, pp. 297-308. https://doi.org/10.1016/j.autcon.2017.12.016
  19. Kim, J., Ham, Y. J., Chung, Y. H. and Chi, S. H. (2019). "Systematic camera placement framework for operation-level visual monitoring on construction jobsites." Journal of Construction Engineering and Management, Vol. 145, No. 4, p. 04019019. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001636
  20. Korea Construction Technology Promotion Act (2016). Enforcement decree article 98 and 99, statutes of the Republic of Korea.
  21. Luo, X., Li, H., Cao, D., Dai, F., Seo, J. and Lee, S. (2018). "Recognizing diverse construction activities in site images via relevance networks of construction-related objects detected by convolutional neural networks." Journal of Computing in Civil Engineering, Vol. 32, No. 3, p. 04018012. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000756
  22. Memarzadeh, M., Golparvar-Fard, M. and Niebles, J. C. (2013). "Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors." Automation in Construction, Vol. 32, pp. 24-37. https://doi.org/10.1016/j.autcon.2012.12.002
  23. Park, M. W. and Brilakis, I. (2012). "Construction worker detection in video frames for initializing vision trackers." Automation in Construction, Vol. 28, pp. 15-25. https://doi.org/10.1016/j.autcon.2012.06.001
  24. Park, M. W. and Brilakis, I. (2016). "Continuous localization of construction workers via integration of detection and tracking." Automation in Construction, Vol. 72, pp. 129-142. https://doi.org/10.1016/j.autcon.2016.08.039
  25. Park, M. W., Elsafty, N. and Zhu, Z. (2015). "Hardhat-wearing detection for enhancing on-site safety of construction workers." Journal of Construction Engineering and Management, Vol. 141, No. 9, p. 04015024. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000974
  26. Ren, S., He, K., Girshick, R. and Sun, J. (2017). "Faster R-CNN: towards real-time object detection with region proposal networks." IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 6, pp. 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031
  27. Settles, B. (2010). Active learning literature survey. Computer Science Technical Report 1648, University of Wisconsin-Madison.
  28. Soltani, M. M., Zhu, Z. and Hammad, A. (2017). "Skeleton estimation of excavator by detecting its parts." Automation in Construction, Vol. 82, pp. 1-15. https://doi.org/10.1016/j.autcon.2017.06.023
  29. Son, H. J., Choi, H. C., Seong, H. W. and Kim, C. W. (2019). "Detection of construction workers under varying poses and changing background in image sequences via very deep residual networks." Automation in Construction, Vol. 99, pp. 27-38. https://doi.org/10.1016/j.autcon.2018.11.033
  30. Yang, J., Vela, P., Teizer, J. and Shi, Z. (2014). "Vision-based tower crane tracking for understanding construction activity." Journal of Computing in Civil Engineering, Vol. 28, No. 1, pp. 103-112. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000242
  31. Zhu, Z., Ren, X. and Chen, Z. (2016). "Visual tracking of construction jobsite workforce and equipment with particle filtering." Journal of Computing in Civil Engineering, Vol. 30, No. 6, pp. 1-15.
  32. Zou, J. and Kim, H. (2007). "Using hue, saturation, and value color space for hydraulic excavator idle time analysis." Journal of Computing in Civil Engineering, Vol. 21, No. 4, pp. 238-246. https://doi.org/10.1061/(ASCE)0887-3801(2007)21:4(238)