The Journal of Korean Institute of Information Technology (한국정보기술학회논문지)

Korean Institute of Information Technology (한국정보기술학회)
권호 표지
DOI QR코드

DOI QR Code

Pavement Crack Detection and Segmentation Based on Deep Neural Network

  • Received : 2019.07.03
  • Accepted : 2019.07.27
  • Published : 2019.09.30
⟨ Previous Next ⟩

Abstract

Cracks on pavement surfaces are critical signs and symptoms of the degradation of pavement structures. Image-based pavement crack detection is a challenging problem due to the intensity inhomogeneity, topology complexity, low contrast, and noisy texture background. In this paper, we address the problem of pavement crack detection and segmentation at pixel-level based on a Deep Neural Network (DNN) using gray-scale images. We propose a novel DNN architecture which contains a modified U-net network and a high-level features network. An important contribution of this work is the combination of these networks afforded through the fusion layer. To the best of our knowledge, this is the first paper introducing this combination for pavement crack segmentation and detection problem. The system performance of crack detection and segmentation is enhanced dramatically by using our novel architecture. We thoroughly implement and evaluate our proposed system on two open data sets: the Crack Forest Dataset (CFD) and the AigleRN dataset. Experimental results demonstrate that our system outperforms eight state-of-the-art methods on the same data sets.

도로 포장면의 크랙(crack)은 도로포장 구조의 열화를 입증하는 중요한 신호와 증상이다. 카메라 영상기반 도로포장 크랙 탐지는 강도 비균질성, 위상 복잡성, 낮은 대조도 및 노이즈성의 텍스처 배경 때문에 어려운 문제이다. 본 논문은 흑백영상에 대하여 깊은 신경망(DNN)에 기반하여 픽셀수준의 도로 크랙 탐지 및 분할 문제에 대해 다룬다. 변형된 U-net 네트워크와 고수준 특징 네트워크를 포함하는 새로운 DNN 구조를 제안한다. 본 연구의 중요 기여는 융합 층을 통해 공급되는 이들 네트워크의 결합 방법이다. 우리가 아는 한, 본 연구는 보도블럭 크랙 분할 및 탐지 문제를 결합을 소개한 최초의 논문이다. 크랙 탐지 및 분할의 시스템 성능은 새로운 구조를 사용하여 급격히 향상되었다. 제안된 시스템을 2개의 공개 데이터셋­크랙 포레스트 데이터셋(CFD)와 AigleRN 데이터셋­에 대하여 구현하고 평가하였다. 본 논문의 시스템은 여덟 가지의 최신 알고리즘과 같은 데이터셋으로 실험을 하였을 때, 가장 뛰어난 결과를 보여주었다.

Keywords

References

  1. A. Mohan and S. Poobal, "Crack detection using image processing: A critical review and analysis", Alexandria Engineering Journal, Vol. 57, No. 2, pp. 787-798, Jun. 2018. https://doi.org/10.1016/j.aej.2017.01.020
  2. O. Ronneberger, P. Fischer, and T. Brox, "U-Nnet: Convolutional networks for biomedical image segmentation", Medical Image Computing and Computer-Assisted Intervention, MICCAI 2015, Springer, LNCS, Vol. 9351, pp. 234-241, May 2015.
  3. P. Liskowski and K. Krawiec, "Segmenting retinal blood vessels with deep neural networks", IEEE Trans Med. Imag., Vol. 35, No. 11, pp. 2369-2380, Nov. 2016. https://doi.org/10.1109/TMI.2016.2546227
  4. V. Badrinarayanan, A. Kendall, and R. Cipolla, "Segnet: A deep convolutional encoder-decoder architecture for image segmentation", IEEE Trans. Pattern Anal. Mach. Intell, Vol. 39, No. 12, pp. 2481-2495, Dec. 2017. https://doi.org/10.1109/TPAMI.2016.2644615
  5. H. Zhao, G. Qin, and X. Wang, "Improvement of canny algorithm based on pavement edge detection", Int. Congress on Image and Signal Processing, Vol. 2, pp. 964-967, Oct. 2010.
  6. H. Oliveira and P. L. Correia, "Automatic road crack segmentation using entropy and image dynamic thresholding", European Signal Processing Conference, Glasgow, UK, pp. 622-626, Aug. 2009.
  7. T. Yamaguchi and S. Hashimoto, "Fast crack detection method for large-size concrete surface images using percolation-based image processing", J. Machine Vision and Applications, Vol. 21, No. 5, pp. 797-809, Aug. 2010. https://doi.org/10.1007/s00138-009-0189-8
  8. Y. Hu and C. Zhao, "A novel LBP based methods for pavement crack detection", Journal of Pattern Recognition Research, Vol. 5, No. 1, pp. 140-147, Jan. 2010. https://doi.org/10.13176/11.167
  9. R. S. Lim, H. M. La, Z. Shan, and W. Sheng, "Developing a crack inspection robot for bridge maintenance", IEEE Int. Conf. on Robotics and Automation, ICRA 2011, Shanghai, China, pp. 6288-6293, May 2011.
  10. R. S. Lim, H. M. La, and W. Sheng, "A robotic crack inspection and mapping system for bridge deck maintenance", IEEE Trans. Autom. Sci. Eng., Vol. 11, No. 2, pp. 367-378, Apr. 2014. https://doi.org/10.1109/TASE.2013.2294687
  11. T. S. Nguyen, S. Begot, F. Duculty, and M. Avila, "Free-form anisotropy: A new method for crack detection on pavement surface images", IEEE Int. Conf. on Image Processing, ICIP 2011, Brussels, Belgium, pp. 1069-1072, Sep. 2011.
  12. M. Avila, S. Begot, F. Duculty, and T.S. Nguyen, "2D image based road pavement crack detection by calculating minimal paths and dynamic programming", IEEE Int. Conf. on Image Processing, ICIP 2014, Paris, France, pp. 783-787, Oct. 2014.
  13. R. Amhaz, S. Chambon, J. Idier, and V. Baltazart, "A new minimal path selection algorithm for automatic crack detection on pavement images", IEEE Int. Conf. on Image Processing, ICIP 2014, Paris, France, pp. 788-792, Oct. 2014.
  14. R. Amhaz, S. Chambon, J. Idier, and V. Baltazart, "Automatic crack detection on two-dimensional pavement images: An algorithm based on minimal path selection", IEEE Trans. Intell. Transp. Syst., Vol. 17, No. 10, pp. 2718-2729, Oct. 2016. https://doi.org/10.1109/TITS.2015.2477675
  15. V. Kaul, A. Yezzi, and Y. Tsai, "Detecting curves with unknown endpoints and arbitrary topology using minimal paths", IEEE Trans. Pattern Anal. Mach. Intell, Vol. 34, No. 10, pp. 1952-1965, Oct. 2012. https://doi.org/10.1109/TPAMI.2011.267
  16. Q. Zou, Y. Cao, Q. Li, Q. Mao, and S. Wang, "Cracktree: Automatic crack detection from pavement images", Pattern Recognition Letters, Vol. 33, No. 3, pp. 227-238, Feb. 2012. https://doi.org/10.1016/j.patrec.2011.11.004
  17. H. Li, D. Song, Y. Liu, and B. Li, "Automatic pavement crack detection by multi-scale image fusion", IEEE Transactions on Intelligent Transportation Systems, Vol. 20, No. 6, pp. 2025-2036, Jun. 2019. https://doi.org/10.1109/tits.2018.2856928
  18. H. Oliveira and P. L. Correia, "Automatic road crack detection and characterization", IEEE Trans. Intell. Transp. Syst., Vol. 14, No. 1, pp. 155-168, Mar. 2013. https://doi.org/10.1109/TITS.2012.2208630
  19. H. Oliveira and P. L. Correia, "CrackIT - an image processing toolbox for crack detection and characterization", IEEE Int. Conf. on Image Processing, ICIP 2014, Paris, France, pp. 798-802, Oct. 2014.
  20. Y. Hu, C. Zhao, and H. Wang, "Automatic pavement crack detection using texture and shape descriptors", Int. J. IETE Technical Review, Vol. 27, No. 5, pp. 398-405, Sep. 2010. https://doi.org/10.4103/0256-4602.62225
  21. K. Fernandes and L. Ciobanu, "Pavement pathologies classification using graph-based features", IEEE Int. Conf. on Image Processing, ICIP 2014, Paris, France, pp. 793-797, Oct. 2014.
  22. D. Ai, G. Jiang, L. Siew Kei, and C. Li, "Automatic pixellevel pavement crack detection using information of multi-scale neighborhoods", IEEE Access, Vol. 6, pp. 24452-24463, Apr. 2018. https://doi.org/10.1109/ACCESS.2018.2829347
  23. Y. Shi, L. Cui, Z. Qi, F. Meng, and Z. Chen, "Automatic road crack detection using random structured forests", IEEE Trans. Intell. Transp. Syst., Vol. 17, No. 12, pp. 3434-3445, Dec. 2016. https://doi.org/10.1109/TITS.2016.2552248
  24. A. Krizhevsky, I. Sutskever, and G.E. Hinton, "ImageNet classification with deep convolutional neural networks", Int. Conf. on Neural Information Processing Systems, NIPS 2012, Vol. 1, pp. 1097-1105, Dec. 2012.
  25. L. Zhang, F. Yang, Y. Daniel Zhang, and Y. J. Zhu, "Road crack detection using deep convolutional neural network", IEEE Int. Conf. on Image Processing, ICIP 2016, Phoenix, AZ, USA, pp. 3708-3712, Sep. 2016.
  26. Y. J. Cha, W. Choi, and O. Buyukozturk, "Deep learning-based crack damage detection using convolutional neural networks", Computer-Aided Civil and Infrastructure Engineering, Vol. 32, No. 5, pp. 361-378, May 2017. https://doi.org/10.1111/mice.12263
  27. K. Gopalakrishnan, S. K. Khaitan, A. Choudhary, and A. Agrawal, "Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection", Int. J. Construction and Building Materials, Vol. 157, pp. 322-330, Dec. 2017. https://doi.org/10.1016/j.conbuildmat.2017.09.110
  28. Z. Fan, Y. Wu, J. Lu, and W. Li, "Automatic pavement crack detection based on structured prediction with the convolutional neural network", CoRR, arXiv:1802.02208v1 [cs.CV] 1 Feb. 2018.
  29. Z. Tong, J. Gao, A. Sha, L. Hu, and S. Li, "Convolutional neural network for asphalt pavement surface texture analysis", Computer-Aided Civil and Infrastructure Engineering, Vol. 33, No. 12, pp. 1056-1072, Dec. 2018. https://doi.org/10.1111/mice.12406
  30. A. Zhang, K. C. P. Wang, B. Li, E. Yang, X. Dai, Y. Peng, Y. Fei, Y. Liu, J. Q. Li, and C. Chen, "Automated pixel-level pavement crack detection on 3d asphalt surfaces using a deep-learning network", Computer-Aided Civil and Infrastructure Engineering, Vol. 32, No. 10, pp. 805-819, Oct. 2017. https://doi.org/10.1111/mice.12297
  31. A. Zhang, K. C. P. Wang, Y. Fei, Y. Liu, C. Chen, G. Yang, J. Q. Li, E. Yang, and S. Qiu, "Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network", Computer-Aided Civil and Infrastructure Engineering, Vol. 34, No. 3, pp. 213-229, Mar. 2019. https://doi.org/10.1111/mice.12409
  32. X. Yang, H. Li, Y. Yu, X. Luo, T. Huang, and X. Yang, "Automatic pixel-level crack detection and measurement using fully convolutional network", Computer-Aided Civil and Infrastructure Engineering, Vol. 33, No. 12, pp. 1090-1109, Aug. 2018. https://doi.org/10.1111/mice.12412
  33. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", CoRR, Sep. 2014.
  34. K. Zuiderveld, "Contrast Limited Adaptive Histogram Equalization", Academic Press Professional, Inc., Graphics gems IV, San Diego, CA, USA, pp. 474-485, 1994.
  35. V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines", Int. Conf. on Machine Learning, ICML 2010, Haifa, Israel, pp. 807-814, Jun. 2010.
  36. D. C. Lee and B. J Park, "Comparison of deep learning activation functions for performance improvement of a 2D shooting game learning agent", Journal of IIBC, Vol. 19, No. 2, pp. 135-141, Apr. 2019.
  37. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift", Int. Conf. on Machine Learning, ICML 2015, Lille, France, Vol. 37, pp. 448-456, Jul. 2015.
  38. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting", Journal of Machine Learning Research, Vol. 15, No. 1, pp. 1929-1958, Jan. 2014.