Browse > Article
http://dx.doi.org/10.7838/jsebs.2021.26.1.093

A Development of Road Crack Detection System Using Deep Learning-based Segmentation and Object Detection  

Ha, Jongwoo (Major in Industrial Data Science & Engineering, Department of Industrial and Data Engineering, Pukyong National University)
Park, Kyongwon (PaveTec Korea Co., Ltd.)
Kim, Minsoo (Major in Industrial Data Science & Engineering, Department of Industrial and Data Engineering, Pukyong National University)
Publication Information
The Journal of Society for e-Business Studies / v.26, no.1, 2021 , pp. 93-106 More about this Journal
Abstract
Many recent studies on deep learning-based road crack detection have shown significantly more improved performances than previous works using algorithm-based conventional approaches. However, many deep learning-based studies are still focused on classifying the types of cracks. The classification of crack types is highly anticipated in that it can improve the crack detection process, which is currently relying on manual intervention. However, it is essential to calculate the severity of the cracks as well as identifying the type of cracks in actual pavement maintenance planning, but studies related to road crack detection have not progressed enough to automated calculation of the severity of cracks. In order to calculate the severity of the crack, the type of crack and the area of the crack in the image must be identified together. This study deals with a method of using Mobilenet-SSD that is deep learning-based object detection techniques to effectively automate the simultaneous detection of crack types and crack areas. To improve the accuracy of object-detection for road cracks, several experiments were conducted to combine the U-Net for automatic segmentation of input image and object-detection model, and the results were summarized. As a result, image masking with U-Net is able to maximize object-detection performance with 0.9315 mAP value. While referring the results of this study, it is expected that the automation of the crack detection functionality on pave management system can be further enhanced.
Keywords
Road Crack Detection; Crack Severity; Object Detection; Crack Segmentation; Pavement Management;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Ashraf, S., Hegazy, I., and Elarif, T. L., "Algorithm for Automatic Crack Analysis and Severity Identification," 2019 IEEE Ninth International Conference on Intelligent Computing and Information Systems, pp. 74-79, 2019.
2 Cubero-Fernandez, A., Rodriguez-Lozano, F. J., Villatoro, R., Olivares, J., and Palomares, J. M., "Efficient pavement crack detection and classification," EURASIP Journal on Image and Video Processing, Vol. 39, pp. 1-11, 2017.
3 Feng, C., Liu, M. Y., Kao, C. C., and Lee, T. Y., "Deep active learning for civil infrastructure defect detection and classification," Computing in Civil Engineering, pp. 298-306, 2017.
4 Gopalakrishnan, K., "Deep Learning in Data-Driven Pavement Image Analysis and Automated Distress Detection: A Review," Data, Vol. 3, No. 3, pp. 1-19, 2018.   DOI
5 Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., and Adam, H., "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704. 04861, 2017.
6 https://github.com/tensorflow/models/tree/master/research/object_detection.
7 Jo, H., Kim, D., Pak, K. W., and Kim, M., "Road damage detection over road scanner images using deep convolutional neural network," ICIC Express Letters, Vol. 14, No. 10, pp. 1001-1008, 2020.   DOI
8 Kim, M., Ryu, J., Cha, D., and Sim, M. K., "Stock Price Prediction Using Sentiment Analysis: from "Stock Discussion Room in Naver," The Journal of Society for e-Business Studies, Vol. 25, No. 4, pp. 61-75, 2020.   DOI
9 Kirillov, A., Girshick, R., He, K., and Dollar, P., "Panoptic Feature Pyramid Networks," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6399-6408, 2019.
10 Koch, C. and Brilakis, I., "Pothole detection in asphalt pavement images," Advanced Engineering Informatics, Vol. 25, No. 3, pp. 507-515, 2011.   DOI
11 Naddaf-Sh, M., Hosseini, S., Zhang, J., Brake, N. A., and Zargarzadeh, H., "Realtime road crack mapping using an optimized convolutional neural network," Complexity, pp. 1-17, 2019.
12 Pauly, L., Hogg, D., Fuentes, R., and Peel, H., "Deeper networks for pavement crack detection," Proceedings of the 34th International Symposium on Automation and Robotics in Construction(ISARC), IAARC, pp. 479-485, 2017.
13 Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. C., "Mobilenetv2: Inverted residuals and linear bottlenecks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
14 Rababaah, H., Vrajitoru, D., and Wolfer, J., "Asphalt pavement crack classification: a comparison of GA, MLP, and SOM," Proceedings of Genetic and Evolutionary Computation Conference, Late-Breaking Paper, 2005.
15 Ragnoli, A., De Blasiis, M. R., and Benedetto, A. D., "Pavement Distress Detection Methods: A Review," MDPI Infrastructure, Vol. 3, No. 58, pp. 1-19, 2018.   DOI
16 Ronneberger, O., Fischer, P., and Brox, T., "U-net: Convolutional networks for biomedical image segmentation," International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, pp. 234-241, 2015.
17 Sorncharean, S. and Phiphobmongkol, S., "Crack detection on asphalt surface image using enhanced grid cell analysis," 4th IEEE International Symposium on Electronic Design, Test and Applications, pp. 49-54, 2008.
18 Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X., and Ling, H., "Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection," IEEE Transactions on Intelligent Transportation Systems, Vol. 21, No. 4, pp. 1525-1535, 2020.   DOI
19 Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., and Berg, A. C., "Ssd: Single shot multibox detector," European Conference on Computer Vision, Springer, Cham, pp. 21-37, 2016.
20 Yoo, W., Seo, J., Kim, D., and Kim, K., "Machine Scheduling Models Based on Reinforcement Learning for Minimizing Due Date Violation and Setup Chang," The Journal of Society for e-Business Studies, Vol. 24, No. 3, pp. 19-33, 2019.
21 Zhang, L., Yang, F., Zhang, Y. D., and Zhu, Y. J., "Road crack detection using deep convolutional neural network," Proceedings of the 2016 IEEE International Conference on Image Processing(ICIP), Phoenix, AZ, USA, 25-28 September 2016, pp. 3708-3712, 2016.
22 Yoo, W., Seo, J., Lee, D., Kim, D., and Kim, K., "Scheduling Generation Model on Parallel Machines with Due Date and Setup Cost Based on Deep Learning," The Journal of Society for e-Business Studies, Vol. 24, No. 3, pp. 99-110, 2019.
23 Zhang, A., Wang, K. C., Li, B., Yang, E., Dai, X., Peng, Y., Fei, Y., Liu, Y., Li, J. Q., and Chen, C., "Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep‐learning network," Computer-Aided Civil and Infrastructure Engineering, Vol. 32, No. 10, pp. 805-819, 2017.   DOI