The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

Metal Surface Defect Detection and Classification using EfficientNetV2 and YOLOv5

EfficientNetV2 및 YOLOv5를 사용한 금속 표면 결함 검출 및 분류

  • ;
  • 김강철 (전남대학교 전기컴퓨터공학부)
  • Received : 2022.06.16
  • Accepted : 2022.08.17
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Detection and classification of steel surface defects are critical for product quality control in the steel industry. However, due to its low accuracy and slow speed, the traditional approach cannot be effectively used in a production line. The current, widely used algorithm (based on deep learning) has an accuracy problem, and there are still rooms for development. This paper proposes a method of steel surface defect detection combining EfficientNetV2 for image classification and YOLOv5 as an object detector. Shorter training time and high accuracy are advantages of this model. Firstly, the image input into EfficientNetV2 model classifies defect classes and predicts probability of having defects. If the probability of having a defect is less than 0.25, the algorithm directly recognizes that the sample has no defects. Otherwise, the samples are further input into YOLOv5 to accomplish the defect detection process on the metal surface. Experiments show that proposed model has good performance on the NEU dataset with an accuracy of 98.3%. Simultaneously, the average training speed is shorter than other models.

철강 표면 결함의 검출 및 분류는 철강 산업의 제품 품질 관리에 중요하다. 그러나 정확도가 낮고 속도가 느리기 때문에 기존 방식은 생산 라인에서 효과적으로 사용할 수 없다. 현재 널리 사용되는 알고리즘(딥러닝 기반)은 정확도 문제가 있으며 아직 개발의 여지가 있다. 본 논문에서는 이미지 분류를 위한 EfficientNetV2와 물체 검출기로 YOLOv5를 결합한 강철 표면 결함 검출 방법을 제안한다. 이 모델의 장점은 훈련 시간이 짧고 정확도가 높다는 것이다. 먼저 EfficientNetV2 모델에 입력되는 이미지는 결함 클래스를 분류하고 결함이 있을 확률을 예측한다. 결함이 있을 확률이 0.3보다 작으면 알고리즘은 결함이 없는 샘플로 인식한다. 그렇지 않으면 샘플이 YOLOv5에 추가로 입력되어 금속 표면의 결함 감지 프로세스를 수행한다. 실험에 따르면 제안된 모델은 NEU 데이터 세트에서 98.3%의 정확도로 우수한 성능을 보였고, 동시에 평균 훈련 속도는 다른 모델보다 단축된 것으로 나타났다.

Keywords

References

  1. K. Song and Y. Yan, "A noise robust method based on completed localbinary patterns for hot-rolled steel strip surface defects," Applied SurfaceScience, vol. 285, Part B, 2013, pp. 858-864.
  2. R. Wei, Y. Song, and Y. Zhang, "Enhanced faster Region Convolutional Neural Networks for Steel Surface Defect Detection," ISIJ Int., vol. 60, issue 3, 2020, pp. 539-545. https://doi.org/10.2355/isijinternational.isijint-2019-335
  3. C. M. Wang, Y. H. Yan, J. Li, H.-Y. Fu, and G.-C. Sun, "Surface quality detection of cold-rolled strip based on BP neural network," MechanicalDesign and Manufacturing, vol. 6, 2007, pp. 106-108.
  4. M. Versaci, S. Calcagno, M. Cacciola, F. C. Morabito, I. Palamara, and D. Pellican'o, "Innovative fuzzy techniques for characterizing defects inultrasonic non-destructive evaluation," Ultrasonic Nondestructive Evaluation Systems, vol. 2014, 2014, pp. 201-232.
  5. Y. Yang and Z. Meng, "Surface defect detection of steel strip based on CNN," Heavy Machinery, vol. 2019, 2019, pp. 25-29.
  6. D. He, K. Xu, and D. Wang, "Design of multi-scale receptive field convolutional neural network for surface inspection of hot-rolled steels," Image and Vision Computing, vol. 89, 2019, pp. 12-20. https://doi.org/10.1016/j.imavis.2019.06.008
  7. Y. He, K. Song, Q. Meng, and Y. Yan, "An end-to-end steel surface defect detection approach via fusing multiple hierarchical features," IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 4, 2020, pp. 1493-1504. https://doi.org/10.1109/tim.2019.2915404
  8. X. Lv, F. Duan, and J. Jiang, "Deep metallic surface defect detection: the new benchmark and detection network," Sensors, vol. 20, no. 6, 2020, p. 1562. https://doi.org/10.3390/s20061562
  9. M. Vannocci, A. Ritacco, and A. Castellano,"Flatness defect detection and classification in hot rolled steel strips using convolutional neural networks," IWANN, vol. 11507, 2019, pp.220-234.
  10. H. Wang, J. Wang, and F. Luo, "Research on surface defect detection of metal sheet and strip based on multi-level feature FasterR-CNN," Mechanical Science and Technology for Aerospace Engineering, vol. 2021, issue 2, 2021, pp. 262-269.
  11. M. Park, B. Kim, and H. Yoon, "A Comparative Study on Machine Learning Models for Red Tide Detection" J. of the Korea Institute off Electronic Communication Sciences, vol. 16, no. 6, Dec. 2021, pp. 1363-1372.
  12. M. Tan Q. Le, "EfficientNetV2: Smaller Models and Faster Training," Proceedings of the 38th International Conference on Machine Learning, PMLR 139:10096-10106, Jun. 2021.
  13. G. Jocher, "Ultralytics/yolov5: v3.1 - Bug Fixes and Performance Improvements," Zenodo, 10.5281/zendo.4154370, Apr. 2021.
  14. G. Lee and M. Moon, "Development a Meal Support for the Visually Impaired Using YOLO Algorithm". J. of the Korea Institute of Electronic Communication Sciences, vol. 16, no. 5, Oct. 2021, pp. 1001-1010. https://doi.org/10.13067/JKIECS.2021.16.5.1001
  15. M. Tan, and Q. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," International Conference on Machine Learning, PMLR 97:6105-6114, May 2020.
  16. Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun. "Yolox: Exceeding yolo series in 2021," arXiv preprint arXiv:2107.08430, 2021.
  17. Z. Tian, C. Shen, H. Chen, and T. He, "Fcos: Fully convolutional one-stage object detection," In Proceedings of the IEEE/CVF international conference on computer vision, Seoul, South Korea, 2019, pp. 9627-9636.
  18. Ch. Wang, A. Bochkovskiy, and M. Liao, "Scaled-yolov4: Scaling cross stage partial network," In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021, pp. 13029-13038.
  19. H. Zhang, Y. Wang, F. Dayoub, and N. Sunderhauf, "Varifocalnet: An iou-aware dense object detector," In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021, pp. 8514-8523.
  20. X. Zhou, V. Koltun, and P. Krahenbuhl, "Probabilistic two-stage detection," arXiv preprint arXiv:2103.07461, 2021.
  21. X. Zhou, D. Wang, and P. Krahenbuhl. "Objects as points." arXiv preprint arXiv:1904.07850 2019.
  22. Z. Yang, S. Liu, H. Hu, L. Wang, and S. Lin, "Reppoints: Point set representation for object detection." In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, South Korea, pp. 9657-9666, 2019.
  23. M. Everingham, S. M. Eslami, L. V. Gool, C. K. Williams, J. Winn, A. Zisserman, "The Pascal Visual Object Classes Challenge: A Retrospective," International Journal of Computer Vision, 111, 2014, pp. 98-136. https://doi.org/10.1007/s11263-014-0733-5
  24. T. Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, "Microsoft coco: Common objects in context," In Proceedings of the 13th European Conference on Computer Cision (ECCV 2014), Zurich, Switzerland, Sept. 2014, pp. 740-755.
  25. R. Xu, H. Lin, K. Lu, L. Cao, and Y. Liu, "A Forest Fire Detection System Based on Ensemble Learning," Forests, vol. 12, 2021, no. 2: 217. https://doi.org/10.3390/f12020217
  26. C. Y. Wang, H. Y. Mark Liao, Y. H. Wu, P. Y. Chen, J. W. Hsieh, and I. H. Yeh, "CSPNet: A new backbone that can enhance learning capability of CNN," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2020), Washington, DC, USA, June 2020; pp. 390-391.
  27. K. Wang, J. H. Liew, Y. Zou, D. Zhou, and J. Feng, "Panet: Few-shot image semantic segmentation with prototype alignment," In Proceedings of the IEEE International Conference on Computer Vision (ICCV2019), Seoul, Korea, Oct. 2019, pp. 9197-9206.
  28. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, and L. Antiga, "Pytorch: An imperative style, high-performance deep learning library," In Proceedings of the Neural Information Processing Systems (NIPS 2019), Vancouver, BC, Canada, Dec. 2019, pp. 8026-8037.
  29. S. Wang, X. Xia, L. Ye, and B. Yang, "Automatic Detection and Classification of Steel Surface Defect Using Deep Convolutional Neural Networks," Metals, vol. 11, 2021, no. 3: 388. https://doi.org/10.3390/met11030388
  30. Y. Xu, K. Zhang, and L. Wang, "Metal Surface Defect Detection Using Modified YOLO," Algorithms, vol. 14, 2021, p. 257. https://doi.org/10.3390/a14090257
  31. Sh. Ren, K. He, R. Girshick, and J. Sun, " Faster R-CNN:Towards real-time object detection with region proposal networks," Advances in neural information processing systems 28, Montreal, Quebec, Canada, Dec. 2015, pp. 91-99.
  32. M. Kim, M. Moon, and Ch. Han, "Expiration Date Notification System Based on YOLO and OCR algorithms for Visually Impaired Person," J. of the Korea Institute of Electronic Communication Sciences, vol. 16, no. 6, Dec. 2021, pp. 1329-1338.