International Journal of Internet, Broadcasting and Communication

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Current Trend and Direction of Deep Learning Method to Railroad Defect Detection and Inspection

  • Han, Seokmin (Major of Data Science, Korea National University of Transportation)
  • Received : 2022.06.19
  • Accepted : 2022.06.24
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

In recent years, the application of deep learning method to computer vision has shown to achieve great performances. Thus, many research projects have also applied deep learning technology to railroad defect detection. In this paper, we have reviewed the researches that applied computer vision based deep learning method to railroad defect detection and inspection, and have discussed the current trend and the direction of those researches. Many research projects were targeted to operate automatically without visual inspection of human and to work in real-time. Therefore, methods to speed up the computation were also investigated. The reduction of the number of learning parameters was considered important to improve computation efficiency. In addition to computation speed issue, the problem of annotation was also discussed in some research projects. To alleviate the problem of time consuming annotation, some kinds of automatic segmentation of the railroad defect or self-supervised methods have been suggested.

Keywords

Acknowledgement

This research was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (NRF-2021R1F1A1049467) and by Korea National University of Transportation in 2022.

References

  1. R. Andersson, Surface defects in rails: potential influence of operational parameters on squat initiation, Thesis for the degree of licentiate of engineering, Department of Applied Mechanics, Chalmers University of Technology, Sweden, 2015.
  2. D. H. Schafer II, Effect of train length on railroad accidents and a quantitative analysis of factors affecting broken rails, B.S. thesis, Dept. Civil Engineering, Univ. of Illinois at Urbana Champaign, IL, USA, 2008.
  3. H. Kim, S. Lee and S. Han, "Railroad Surface Defect Segmentation Using a Modified Fully Convolutional Network," KSII Transactions on Internet and Information Systems, Korean Society for Internet Information (KSII), Vol.14, No.12, pp. 4763-4775, Dec 2020. DOI: https://doi.org/10.3837/tiis.2020.12.008.
  4. L. Xiao, B. Wu, and Y. Hu, "Surface defect detection using image pyramid," IEEE Sensors Journal, Vol. 20, pp. 7181-7188, 2020. DOI: https://doi.org/10.1109/JSEN.2020.2977366.
  5. D. Huang, S. Liao, A. I. Sunny, and S. Yu, "A novel automatic surface scratch defect detection for fluid-conveying tube of Coriolis mass flow-meter based on 2D-direction filter," Measurement, Vol. 126, pp. 332-341 2018. DOI: https://doi.org/10.1016/j.measurement.2018.05.060.
  6. J. Wang, L. Luo, W. Ye, and S. Zhu, "A defect-detection method of split pins in the catenary fastening devices of highspeed railway based on deep learning," IEEE Transactions on Instrumentation and Measurement, IEEE Transactions on Instrumentation and Measurement, Vol. 69, No. 12, pp. 9517-9525, Dec 2020. DOI: https://doi.org/10.1109/TIM.2020.3006324.
  7. L. Peng, S. Zheng, P. Li, Y. Wang, and Q. Zhong, "A comprehensive detection system for track geometry using fused vision and inertia," IEEE Transactions on Instrumentation and Measurement, Vol. 70, pp. 1-15, 2021. DOI: https://doi.org/10.1109/TIM.2020.3039301.
  8. H. Zhang, X. Jin, Q. M. J. Wu, Y. Wang, Z. He and Y. Yang, "Automatic Visual Detection System of Railway Surface Defects With Curvature Filter and Improved Gaussian Mixture Model," IEEE Transactions on Instrumentation and Measurement, vol. 67, no. 7, pp. 1593-1608, July 2018. DOI: https://doi.org/10.1109/tim.2018.2803830.
  9. Y. Min, B. Xiao, and J. Dang, "Real time detection system for rail surface defects based on machine vision," J Image Video Proc., 2018. DOI: https://doi.org/10.1186/s13640-017-0241-y.
  10. Y. Wu, Y. Qin, Z. Wang, and L. Jia, "A UAV-Based Visual Inspection Method for Rail Surface Defects," Applied Sciences, Vol. 8, No. 7, pp. 1028, Jun. 2018. DOI: https://doi.org/10.3390/app8071028.
  11. Z. He, Y. Wang, F. Yin, and J. Liu, "Surface defect detection for high-speed rails using an inverse P-M diffusion model", Sensor Review, Vol. 36 No. 1, pp. 86-97, 2016. DOI: https://doi.org/10.1108/SR-03-2015-0039.
  12. J. Gan, Q. Li, J. Wang and H. Yu, "A Hierarchical Extractor-Based Visual Rail Surface Inspection System," IEEE Sensors Journal, Vol. 17, No. 23, pp. 7935-7944, Dec 2017. DOI: https://doi.org/10.1109/JSEN.2017.2761858
  13. D. Zheng, L. Li, S. Zheng, X. Chai, S. Zhao, Q. Tong, J. Wang, and L. Guo, "A Defect Detection Method for Rail Surface and Fasteners Based on Deep Convolutional Neural Network", Computational Intelligence and Neuroscience, Vol. 2021, 2021. DOI: https://doi.org/10.1155/2021/2565500
  14. Glenn Jocher, "ultralytics/yolov5: v6.1 - TensorRT, TensorFlow Edge TPU and OpenVINO Export and Inference". Zenodo, Feb 2022, DOI: https://doi.org/10.5281/zenodo.6222936
  15. K. He, G. Gkioxari, P. Dollar and R. Girshick, "Mask R-CNN," 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980-2988, 2017. DOI: https://doi.org/10.1109/ICCV.2017.322.
  16. K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90.
  17. J. H. Feng, H. Yun, Y. Q. Hu, J. Lin, S. W. Liu, X. Luo, "Research on deep learning method for rail surface defect detection," IET Electrical Systems in Transportation, Vol. 10, pp. 436-442, Dec 2020. DOI: https://doi.org/10.1049/ietest.2020.0041.
  18. T. Bai, J. Gao, J. Yang, and D. Yao, "A Study on Railway Surface Defects Detection Based on Machine Vision," Entropy, Vol. 23, No. 11, pp. 1437, Oct 2021. DOI: https://doi.org/10.3390/e23111437.
  19. Y. Liu, H. Xiao, J. Xu and J. Zhao, "A Rail Surface Defect Detection Method Based on Pyramid Feature and Lightweight Convolutional Neural Network," IEEE Transactions on Instrumentation and Measurement, Vol. 71, pp. 1-10, 2022. DOI: https://doi.org/10.1109/TIM.2022.3165287.
  20. H. Yang, Y. Wang, J. Hu, J. He, Z. Yao and Q. Bi, "Deep Learning and Machine Vision-Based Inspection of Rail Surface Defects," IEEE Transactions on Instrumentation and Measurement, Vol. 71, pp. 1-14, 2022. DOI: https://doi.org/10.1109/TIM.2021.3138498.
  21. Y. Boykov and G. Funka-Lea, "Graph cuts and effcient N-D image segmentation," Int J Comput Vision, Vol. 70, pp. 109-131, 2006. DOI: https://doi.org/10.1007/s11263-006-7934-5.
  22. X. Giben, V. M. Patel and R. Chellappa, "Material classification and semantic segmentation of railway track images with deep convolutional neural networks," 2015 IEEE International Conference on Image Processing (ICIP), pp. 621-625, 2015. DOI: https://doi.org/10.1109/ICIP.2015.7350873.
  23. S. Faghih-Roohi, S. Hajizadeh, A. Nunez, R. Babuska and B. De Schutter, "Deep convolutional neural networks for detection of rail surface defects," 2016 InternationalJoint Conference on Neural Networks(IJCNN), pp. 2584-2589, 2016. DOI: https://doi.org/10.1109/IJCNN.2016.7727522.
  24. P. Bojarczak and P. Lesiak, "UAVs in rail damage image diagnostics supported by deep-learning networks," Open Engineering, Vol. 11, No. 1, pp. 339-348, Jan 2021. DOI: https://doi.org/10.1515/eng-2021-0033
  25. J. Long, E. Shelhamer and T. Darrell, "Fully convolutional networks for semantic segmentation," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298965.
  26. X. Pan, and T. Y. Yang, "Image-based monitoring of bolt loosening through deep learning based integrated detection and tracking," Computer-Aided Civil and Infrastructure Engineering, Vol. 37, No. 10, pp. 1207-1222, Jun 2021. DOI : https://doi.org/10.1111/mice.12710.
  27. J. Liu, Y. Huang, S. Wang, X. Zhao, Q. Zou and X. Zhang, "Rail fastener defect inspection method for multi railways based on machine vision," Railway Sciences, May 2022. DOI: https://doi.org/10.1108/RS-04-2022-0012
  28. Y. Min and Y. Li, "Self-Supervised Railway Surface Defect Detection with Defect Removal Variational Autoencoders," Energies, Vol. 15, No. 10, pp. 3592, May 2022. DOI: https://doi.org/10.3390/en15103592.