The Journal of The Korea Institute of Intelligent Transport Systems (한국ITS학회 논문지)

The Korea Institute of Intelligent Transport Systems (한국ITS학회)
권호 표지
DOI QR코드

DOI QR Code

Detection Algorithm of Road Surface Damage Using Adversarial Learning

적대적 학습을 이용한 도로 노면 파손 탐지 알고리즘

  • Shim, Seungbo (Korea Institute of Civil Engineering and Building Technology)
  • 심승보 (한국건설기술연구원 인프라안전연구본부)
  • Received : 2021.06.13
  • Accepted : 2021.07.01
  • Published : 2021.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Road surface damage detection is essential for a comfortable driving environment and the prevention of safety accidents. Road management institutes are using automated technology-based inspection equipment and systems. As one of these automation technologies, a sensor to detect road surface damage plays an important role. For this purpose, several studies on sensors using deep learning have been conducted in recent years. Road images and label images are needed to develop such deep learning algorithms. On the other hand, considerable time and labor will be needed to secure label images. In this paper, the adversarial learning method, one of the semi-supervised learning techniques, was proposed to solve this problem. For its implementation, a lightweight deep neural network model was trained using 5,327 road images and 1,327 label images. After experimenting with 400 road images, a model with a mean intersection over a union of 80.54% and an F1 score of 77.85% was developed. Through this, a technology that can improve recognition performance by adding only road images was developed to learning without label images and is expected to be used as a technology for road surface management in the future.

도로 노면 파손 탐지는 쾌적한 주행 환경과 안전사고의 예방을 위해 필요하다. 도로 관리 기관은 자동화 기술 기반의 검사 장비와 시스템을 활용하고 있다. 이러한 자동화 기술 중에서도 도로 노면의 파손을 탐지하는 기술은 중요한 역할을 수행한다. 최근 들어 딥러닝을 이용한 기술에 대한 연구가 활발하게 진행 중이다. 이러한 딥러닝 기술 개발을 위해서는 도로 영상과 라벨 영상이 필요하다. 하지만 라벨 영상을 확보하기 위해서는 많은 시간과 노동력이 요구된다. 본 논문에서는 이러한 문제를 해결하기 위하여 준지도 학습 기법 중 하나인 적대적 학습 방법을 제안했다. 이를 구현하기 위해서 5,327장의 도로 영상과 1,327장의 라벨 영상을 사용하여 경량화 심층 신경망 모델을 학습했다. 그리고 이를 400장의 도로 영상으로 실험한 결과 80.54%의 mean intersection over union과 77.85%의 F1 score를 갖는 모델을 개발하였다. 결과적으로 라벨 영상 없이 도로 영상만을 학습에 추가하여 인식 성능을 향상시킬 수 있는 기술을 개발하였고, 향후 도로 노면 관리를 위한 기술로 활용되길 기대한다.

Keywords

Acknowledgement

본 연구는 한국건설기술연구원 주요사업 "이종 데이터 변환을 통한 준지도 학습 기반 균열 탐지 기술 개 발"의 연구비 지원에 의해 수행되었습니다.

References

  1. Bang S., Park S., Kim H. and Kim H.(2018), "A deep residual network with transfer learning for pixel-level road crack detection," In Proc. International Symposium on Automation and Robotics in Construction, Berlin, Germany, vol. 35, pp.1-4.
  2. Bang S., Park S., Kim H. and Kim H.(2019), "Encoder-decoder network for pixel level road crack detection in black-box images," Computer-Aided Civil and Infrastructure Engineering, vol. 34, no. 8, pp.713-727. https://doi.org/10.1111/mice.12440
  3. Fan R.(2018), Real-time computer stereo vision for automotive applications, Doctoral Dissertation, University of Bristol.
  4. Fan Z., Li C., Chen Y., Wei J., Loprencipe G., Chen X. and Di Mascio P.(2020), "Automatic crack detection on road pavements using encoder-decoder architecture," Materials, vol. 13, p.2960. https://doi.org/10.3390/ma13132960
  5. Feng H., Xu G. S. and Guo Y.(2018), "Multi-scale classification network for road crack detection," IET Intelligent Transport Systems, vol. 13, no. 2, pp.398-405. https://doi.org/10.1049/iet-its.2018.5280
  6. Goodfellow I. J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A. and Bengio Y.(2014), Generative adversarial networks [Online], arXiv:1406.2661. Available at https://arxiv.org/abs/1406.2661
  7. Guan H., Li J., Yu Y., Chapman M. and Wang C.(2014), "Automated road information extraction from mobile laser scanning data," IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 1, pp.194-205. https://doi.org/10.1109/TITS.2014.2328589
  8. He K., Zhang X., Ren S. and Sun J.(2016), "Deep residual learning for image recognition," In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp.770-778.
  9. Huang G., Liu Z., Van Der Maaten L. and Weinberger K. Q.(2017), "Densely connected convolutional networks," In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp.4700-4708.
  10. Hung W. C., Tsai Y. H., Liou Y. T., Lin Y. Y. and Yang M. H.(2018), Adversarial learning for semi-supervised semantic segmentation [Online], arXiv:1802.07934. Available at https://arxiv.org/abs/1802.07934
  11. Jenkins M. D., Carr T. A., Iglesias M. I., Buggy T. and Morison G.(2018), "A deep convolutional neural network for semantic pixel-wise segmentation of road and pavement surface cracks," In Proc. 26th European Signal Processing Conference(EUSIPCO), Rome, Italy pp.2120-2124.
  12. Kingma D. P. and Ba J.(2014), Adam: A method for stochastic optimization [Online], arXiv:1412.6980. Available at https://arxiv.org/abs/1412.6980
  13. Laurent J., Hebert J. F., Lefebvre D. and Savard Y.(2012), "Using 3D laser profiling sensors for the automated measurement of road surface conditions," In Proc. 7th RILEM International Conference on Cracking in Pavements, Delft, Netherlands, pp.157-167.
  14. Li G., Wan J., He S., Liu Q. and Ma B.(2020), "Semi-supervised semantic segmentation using adversarial learning for pavement crack detection," IEEE Access, vol. 8, pp.51446-51459. https://doi.org/10.1109/ACCESS.2020.2980086
  15. Madli R., Hebbar S., Pattar P. and Golla V.(2015), "Automatic detection and notification of potholes and humps on roads to aid drivers," IEEE Sensors Journal, vol. 15, no. 8, pp.4313-4318. https://doi.org/10.1109/JSEN.2015.2417579
  16. Mei Q., Gul M. and Azim M. R.(2020), "Densely connected deep neural network considering connectivity of pixels for automatic crack detection," Automation in Construction, vol. 110, p.103018. https://doi.org/10.1016/j.autcon.2019.103018
  17. Ouali Y., Hudelot C. and Tami M.(2020), An overview of deep semi-supervised learning [Online], arXiv:2006.05278. Available at https://arxiv.org/abs/2006.05278
  18. Romera E., Alvarez J. M., Bergasa L. M. and Arroyo R.(2017), "Erfnet: Efficient residual factorized convnet for real-time semantic segmentation," IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 1, pp.263-272. https://doi.org/10.1109/tits.2017.2750080
  19. Shi Y., Cui L., Qi Z., Meng F. and Chen Z.(2016), "Automatic road crack detection using random structured forests," IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 12, pp.3434-3445. https://doi.org/10.1109/TITS.2016.2552248
  20. Shim S., Kim J., Cho G. C. and Lee S. W.(2020), "Multiscale and adversarial learning-based semi-supervised semantic segmentation approach for crack detection in concrete structures," IEEE Access, vol. 8, pp.170939-170950. https://doi.org/10.1109/ACCESS.2020.3022786
  21. Singla A., Bertino E. and Verma D.(2019), "Overcoming the lack of labeled data: Training intrusion detection models using transfer learning," In Proc. IEEE International Conference on Smart Computing(SMARTCOMP), Washington, DC, USA, pp.69-74.
  22. Zhang A., Wang K. C. P., Fei Y., Liu Y., Chen C., Yang G., Li J. Q., Yang E. and Qiu S.(2019), "Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network," Computer-Aided Civil and Infrastructure Engineering, vol. 34, no. 3, pp.213-229. https://doi.org/10.1111/mice.12409
  23. Zhang Y., Wang S., Chen B., Cao J. and Huang Z.(2021), "TrafficGAN: Network-scale deep traffic prediction with generative adversarial nets," IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 1, pp.219-230. https://doi.org/10.1109/TITS.2019.2955794
  24. Zou Q., Zhang Z., Li Q., Qi X., Wang Q. and Wang S.(2018), "Deepcrack: Learning hierarchical convolutional features for crack detection," IEEE Transactions on Image Processing, vol. 28, no. 3, pp.1498-1512. https://doi.org/10.1109/tip.2018.2878966