Browse > Article
http://dx.doi.org/10.12652/Ksce.2019.39.5.0631

Automated Vision-based Construction Object Detection Using Active Learning  

Kim, Jinwoo (Seoul National University)
Chi, Seokho (Seoul National University)
Seo, JoonOh (Hong Kong Polytechnic University)
Publication Information
KSCE Journal of Civil and Environmental Engineering Research / v.39, no.5, 2019 , pp. 631-636 More about this Journal
Abstract
Over the last decade, many researchers have investigated a number of vision-based construction object detection algorithms for the purpose of construction site monitoring. However, previous methods require the ground truth labeling, which is a process of manually marking types and locations of target objects from training image data, and thus a large amount of time and effort is being wasted. To address this drawback, this paper proposes a vision-based construction object detection framework that employs an active learning technique while reducing manual labeling efforts. For the validation, the research team performed experiments using an open construction benchmark dataset. The results showed that the method was able to successfully detect construction objects that have various visual characteristics, and also indicated that it is possible to develop the high performance of an object detection model using smaller amount of training data and less iterative training steps compared to the previous approaches. The findings of this study can be used to reduce the manual labeling processes and minimize the time and costs required to build a training database.
Keywords
Construction object; Vision-based; Object detection; Active learning; Computer vision;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Azar, E. R. and McCabe, B. (2012a). "Automated visual recognition of dump trucks in construction videos." Journal of Computing in Civil Engineering, Vol. 26, No. 6, pp. 769-781.   DOI
2 Azar, E. R. and McCabe, B. (2012b). "Part based model and spatial-temporal reasoning to recognize hydraulic excavators in construction images and videos." Automation in Construction, Vol. 24, pp. 194-202.   DOI
3 Azar, E. R., Dickinson, S. and McCabe, B. (2013). "Server-Customer interaction tracker: computer vision-based system to estimate dirt-loading cycles." Journal of Construction Engineering and Management, Vol. 139, No. 7, pp. 785-794.   DOI
4 Cai, J., Zhang, Y. and Cai, H. (2019). "Two-step long short-term memory method for identifying construction activities through positional and attentional cues." Automation in Construction, Vol. 106, p. 102886.   DOI
5 Chi, S. and Caldas, C. H. (2011). "Automated object identification using optical video cameras on construction sites." Computer-Aided Civil and Infrastructure Engineering, Vol. 26, No. 5, pp. 368-380.   DOI
6 Chi, S. and Caldas, C. H. (2012). "Image-based safety assessment: automated spatial safety risk identification of earthmoving and surface mining activities." Journal of Construction Engineering and Management, Vol. 138, No. 3, pp. 341-351.   DOI
7 Chi, S., Caldas, C. H. and Kim, D. Y. (2009). "A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques." Computer-Aided Civil and Infrastructure Engineering, Vol. 24, No. 3, pp. 199-211.   DOI
8 Chung, S. (2018). Bridge damage factor recognition from inspection reports using active recurrent neural network, Master Thesis, Seoul National University.
9 Fang, W., Ding, L., Zhong, B., Love, P. E. D. and Luo, H. (2018). "Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach." Advanced Engineering Informatics, Vol. 37, pp. 139-149.   DOI
10 Golparvar-Fard, M., Heydarian, A. and Niebles, J. C. (2013). "Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers." Advanced Engineering Informatics, Vol. 27, No. 4, pp. 652-663.   DOI
11 Gong, J. and Caldas, C. H. (2010). "Computer vision-based video interpretation model for automated productivity analysis of construction operations." Journal of Computing in Civil Engineering, Vol. 24, No. 3, pp. 252-263.   DOI
12 Han, S. U., Achar, M., Lee, S. H. and Pena-Mora, F. (2013). "Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring." Visualization in Engineering, Vol. 1, No. 1, pp. 1-13.   DOI
13 Han, S. U., Lee, S. H. and Pena-mora, F. (2014). "Vision-based detection of unsafe actions of a construction worker: case study of ladder climbing." Journal of Computing in Civil Engineering, Vol. 27, No. 6, pp. 635-644.   DOI
14 Kim, H. J., Kim, H. K., Hong, Y. W. and Byun, H. R. (2018b). "Detecting construction equipment using a region-based fully convolutional network and transfer learning." Journal of Computing in Civil Engineering, Vol. 32, No. 2, p. 04017082.   DOI
15 Kim, H. J., Kim, K. N. and Kim, H. K. (2016). "Vision-based object-centric safety assessment using fuzzy inference: monitoring struck-by accidents with moving objects." Journal of Computing in Civil Engineering, Vol. 30, No. 4, p. 04015075.   DOI
16 Kim, J. W. and Chi, S. H. (2017). "Adaptive detector and tracker on constrution sites using functional integration and online learning." Journal of Computing in Civil Engineering, Vol. 31, No. 5, p. 04017026.   DOI
17 Kim, J. W. and Chi, S. H. (2019). "Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles." Automation in Construction, Vol. 104, pp. 255-264.   DOI
18 Kim, J. W., Chi, S. H. and Seo, J. W. (2018a). "Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks." Automation in Construction, Vol. 87, pp. 297-308.   DOI
19 Kim, J., Ham, Y. J., Chung, Y. H. and Chi, S. H. (2019). "Systematic camera placement framework for operation-level visual monitoring on construction jobsites." Journal of Construction Engineering and Management, Vol. 145, No. 4, p. 04019019.   DOI
20 Korea Construction Technology Promotion Act (2016). Enforcement decree article 98 and 99, statutes of the Republic of Korea.
21 Luo, X., Li, H., Cao, D., Dai, F., Seo, J. and Lee, S. (2018). "Recognizing diverse construction activities in site images via relevance networks of construction-related objects detected by convolutional neural networks." Journal of Computing in Civil Engineering, Vol. 32, No. 3, p. 04018012.   DOI
22 Memarzadeh, M., Golparvar-Fard, M. and Niebles, J. C. (2013). "Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors." Automation in Construction, Vol. 32, pp. 24-37.   DOI
23 Park, M. W. and Brilakis, I. (2012). "Construction worker detection in video frames for initializing vision trackers." Automation in Construction, Vol. 28, pp. 15-25.   DOI
24 Park, M. W. and Brilakis, I. (2016). "Continuous localization of construction workers via integration of detection and tracking." Automation in Construction, Vol. 72, pp. 129-142.   DOI
25 Park, M. W., Elsafty, N. and Zhu, Z. (2015). "Hardhat-wearing detection for enhancing on-site safety of construction workers." Journal of Construction Engineering and Management, Vol. 141, No. 9, p. 04015024.   DOI
26 Ren, S., He, K., Girshick, R. and Sun, J. (2017). "Faster R-CNN: towards real-time object detection with region proposal networks." IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 6, pp. 1137-1149.   DOI
27 Settles, B. (2010). Active learning literature survey. Computer Science Technical Report 1648, University of Wisconsin-Madison.
28 Soltani, M. M., Zhu, Z. and Hammad, A. (2017). "Skeleton estimation of excavator by detecting its parts." Automation in Construction, Vol. 82, pp. 1-15.   DOI
29 Son, H. J., Choi, H. C., Seong, H. W. and Kim, C. W. (2019). "Detection of construction workers under varying poses and changing background in image sequences via very deep residual networks." Automation in Construction, Vol. 99, pp. 27-38.   DOI
30 Yang, J., Vela, P., Teizer, J. and Shi, Z. (2014). "Vision-based tower crane tracking for understanding construction activity." Journal of Computing in Civil Engineering, Vol. 28, No. 1, pp. 103-112.   DOI
31 Zhu, Z., Ren, X. and Chen, Z. (2016). "Visual tracking of construction jobsite workforce and equipment with particle filtering." Journal of Computing in Civil Engineering, Vol. 30, No. 6, pp. 1-15.
32 Zou, J. and Kim, H. (2007). "Using hue, saturation, and value color space for hydraulic excavator idle time analysis." Journal of Computing in Civil Engineering, Vol. 21, No. 4, pp. 238-246.   DOI