The Journal of Bigdata (한국빅데이터학회지)

Korea Bigdata Society (한국빅데이터학회)
권호 표지
DOI QR코드

DOI QR Code

Real-time Steel Surface Defects Detection Appliocation based on Yolov4 Model and Transfer Learning

Yolov4와 전이학습을 기반으로한 실시간 철강 표면 결함 검출 연구

  • 김복경 (부경대학교 기술경영협동과정) ;
  • 배준희 (부경대학교 기술경영협동과정) ;
  • 환 ((주)인타운) ;
  • 이용은 ((주)인타운) ;
  • 옥영석 (부경대학교 기술경영전문대학원)
  • Received : 2022.11.14
  • Accepted : 2022.12.16
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Steel is one of the most fundamental components to mechanical industry. However, the quality of products are greatly impacted by the surface defects in the steel. Thus, researchers pay attention to the need for surface defects detector and the deep learning methods are the current trend of object detector. There are still limitations and rooms for improvements, for example, related works focus on developing the models but don't take into account real-time application with practical implication on industrial settings. In this paper, a real-time application of steel surface defects detection based on YOLOv4 is proposed. Firstly, as the aim of this work to deploying model on real-time application, we studied related works on this field, particularly focusing on one-stage detector and YOLO algorithm, which is one of the most famous algorithm for real-time object detectors. Secondly, using pre-trained Yolov4-Darknet platform models and transfer learning, we trained and test on the hot rolled steel defects open-source dataset NEU-DET. In our study, we applied our application with 4 types of typical defects of a steel surface, namely patches, pitted surface, inclusion and scratches. Thirdly, we evaluated YOLOv4 trained model real-time performance to deploying our system with accuracy of 87.1 % mAP@0.5 and over 60 fps with GPU processing.

철강은 기계 산업의 가장 기본적인 구성 요소 중 하나이다. 그러나 철강의 표면 결함은 제품의 품질에 큰 영향을 미친다. 따라서 연구자들은 표면 결함 감지기의 필요성에 주목하고 딥 러닝을 이용한 방법은 객체 결함 감지를 하는데 많이 사용된다. 연구 개발용으로 학습 모델 개발에 초점을 맞추지만 실제 산업환경에 실질적인 영향을 미치는 실시간 적용은 아직 적용되지 않는 한계와 개선의 여지가 필요하다. 본 연구는 YOLOv4를 기반으로 한 철강 표면 결함 감지의 실시간 적용을 제안한다. 첫째, 본 연구는 실시간 응용 모델을 적용하는 것을 목적으로 하며 실시간 객체 검출기의 가장 유명한 알고리즘 중 하나인 one-stage Detector의 YOLO 알고리즘을 중심으로 연구를 진행하였다. 둘째, 사전 훈련된 YOLOv4-Darknet 플랫폼 모델과 전이학습을 사용하여 철강 표면 오픈 소스 데이터셋 NEU-DET을 이용하여 학습과 테스트를 진행하였다. 본 연구에서는 철강 표면의 패치, 구멍 난 표면, 불순물, 스크래치 4가지 유형의 결함을 이용하였다. 셋째, 87.1% mAP@0.5의 정확도와 60fps 이상의 시스템 구축을 위해 YOLOv4를 이용하여 훈련된 모델의 실시간 성능을 평가하였다.

Keywords

Acknowledgement

This work was supported by the Technology Development Program(S3125098) funded by the Ministry of SMEs and Startups (MSS, Republic of Korea).

References

  1. Xing, J.; Jia, M. A convolutional neural network-based method for workpiece surface defect detection. Measurement 2021,176, 109185.
  2. S. Ren, K. He, R. Girshick and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks" in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 39, no. 06, pp. 1137-1149, 2017. https://doi.org/10.1109/TPAMI.2016.2577031
  3. Cheng, J.C.; Wang, M. Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Autom. Constr. 2018, 95, 155-171. https://doi.org/10.1016/j.autcon.2018.08.006
  4. Li, R.; Yuan, Y.; Zhang, W.; Yuan, Y. Unified vision-based methodology for simultaneous concrete defect detection and geolocalization. Comput.-Aided Civ. Infrastruct. Eng. 2018, 33, 527-544. https://doi.org/10.1111/mice.12351
  5. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," Computer Vision - ECCV 2016. pp. 21-37.
  6. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June-1 July 2016; pp. 779-788.
  7. Zhang, C.; Chang, C.C.; Jamshidi, M. Bridge damage detection using a single-stage detector and field inspection images. arXiv 2018, arXiv:1812.10590
  8. Yin, X.; Chen, Y.; Bouferguene, A.; Zaman, H.; Al-Hussein, M.; Kurach, L. A deep learning-based framework for an automated defect detection system for sewer pipes. Autom. Constr. 2020, 109, 102967.
  9. Deng J, Lu Y, Lee VC-S. Imaging-based crack detection on concrete surfaces using You Only Look Once network. Structural Health Monitoring. 2021;20(2):484-499. https://doi.org/10.1177/1475921720938486
  10. Jiaqiao Zhang, Xin Kang, Hongjun Ni & Fuji Ren Surface defect detection of steel strips based on classification priority YOLOv3-dense network, Ironmaking & Steelmaking, 48:5, 547-558, DOI: 2021. https://doi.org/10.1080/03019233.2020.1816806
  11. Lv, X.; Duan, F.; Jiang, J.-j.; Fu, X.; Gan, L. Deep Metallic Surface Defect Detection: The New Benchmark and Detection Network. 2020, 1562.
  12. Redmon, J.; Farhadi, A. Yolov3: An incremental improvement. arXiv 2018, arXiv:1804.02767.
  13. Joseph Redmon and Ali Farhadi. YOLO9000: Better, Faster, Stronger.,CVPR2017,.6517-6525, 2017.
  14. Bochkovskiy, A., C.-Y. Wang, and H.-Y.M. Liao YOLOv4: Optimal Speed and Accuracy of Object Detection. 2020. arXiv:2004.10934.
  15. YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet). Available online: https://github.com/AlexeyAB/darknet (accessed on 24 Feb 2022).
  16. http://faculty.neu.edu.cn/yunhyan/NEU_surface_defect_database.html.(accessed on 24 Feb 2022).
  17. J. Li, Z. Su, J. Gen g, an d Y. Yin , "Real-time detection of steel strip surface defects based on improved YOLO detection network," IFAC-PapersOnLine, vol. 51, no. 21, pp. 76-81, 2018.