Journal of Korean Society of Industrial and Systems Engineering (산업경영시스템학회지)

Society of Korea Industrial and System Engineering (한국산업경영시스템학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Image Annotation Automation Process using SHAP for Defect Detection

SHAP를 이용한 이미지 어노테이션 자동화 프로세스 연구

  • Jin Hyeong Jung (Department of Industrial and Systems Engineering, Kyonggi University Graduate School) ;
  • Hyun Su Sim (Department of Industrial and Systems Engineering, Kyonggi University Graduate School) ;
  • Yong Soo Kim (Department of Industrial and Systems Engineering, Kyonggi University)
  • 정진형 (경기대학교 일반대학원 산업시스템공학과) ;
  • 심현수 (경기대학교 일반대학원 산업시스템공학과) ;
  • 김용수 (경기대학교 산업시스템공학과)
  • Received : 2023.03.03
  • Accepted : 2023.03.20
  • Published : 2023.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, the development of computer vision with deep learning has made object detection using images applicable to diverse fields, such as medical care, manufacturing, and transportation. The manufacturing industry is saving time and money by applying computer vision technology to detect defects or issues that may occur during the manufacturing and inspection process. Annotations of collected images and their location information are required for computer vision technology. However, manually labeling large amounts of images is time-consuming, expensive, and can vary among workers, which may affect annotation quality and cause inaccurate performance. This paper proposes a process that can automatically collect annotations and location information for images using eXplainable AI, without manual annotation. If applied to the manufacturing industry, this process is thought to save the time and cost required for image annotation collection and collect relatively high-quality annotation information.

Keywords

Acknowledgement

This work was supported by Korea Institute for Advancement of Technology(KIAT) grant funded by the Korea Government(MOTIE) (P0008691, HRD Program for Industrial Innovation)

References

  1. Chen, Y., Zeng, X., Chen, X., and Guo, W., A Survey on Automatic Image Annotation, Applied Intelligence, 2020, Vol. 50, pp. 3412-3428. https://doi.org/10.1007/s10489-020-01696-2
  2. Casting Product Image Data for Quality Inspection, https://www.kaggle.com/ravirajsinh45/real-life-industrial-dataset-of-casting-product, 2020.
  3. Du, W., Shen, H., Fu, J., Zhang, G., and He, Q. Approaches for Improvement of the X-ray Image Defect Detection of Automobile Casting Aluminum Parts Based on Deep Learning, NDT & E International, 2019, 107, p. 102144.
  4. Ferguson, M., Ak, R., Lee, Y.T.T., and Law, K.H., Automatic Localization of Casting Defects with Convolutional Neural Networks, In 2017 IEEE international conference on big data (big data), 2017, Boston, USA, pp. 1726-1735.
  5. Hanbury, A., A Survey of Methods for Image Annotation, Journal of Visual Languages & Computing, 2018, Vol. 19, No. 5, pp. 617-627. https://doi.org/10.1016/j.jvlc.2008.01.002
  6. Hu, C. and Wang, Y., An Efficient Convolutional Neural Network Model Based on Object-Level Attention Mechanism for Casting Defect Detection on Radiography Images, IEEE Transactions on Industrial Electronics, 2020, Vol. 67, No. 12, pp. 10922-10930. https://doi.org/10.1109/tie.2019.2962437
  7. Jiang, L., Wang, Y., Tang, Z., Miao, Y., and Chen, S., Casting Defect Detection in X-ray Images Using Convolutional Neural Networks and Attention-guided Data Augmentation, Measurement, 2021, Vol. 170, p. 108736.
  8. Ling, H., Gao, J., Kar, A., Chen, W., and Fidler, S., Fast Interactive Object Annotation with Curve-GCN, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, Long Beach, USA, pp. 5257-5266.
  9. Liong, S.T., Gan, Y.S., Huang, Y.-C., Yuan, C.A. and Chang, H.-C., Automatic Defect Segmentation on Leather with Deep Learning, arXiv:1903.12139, 2019, https://doi.org/10.48550/arXiv.1903.12139.
  10. Ma, J. and Yuan, Y., Dimension Reduction of Image Deep Feature Using PCA, Journal of Visual Communication and Image Representation, 2019, Vol. 63, p. 102578.
  11. Ma, Y., Liu, Y., Xie, Q., and Li, L., CNN-feature Based Automatic Image Annotation Method, Multimedia Tools and Applications, 2019, Vol. 78, No. 3, pp. 3767-3780. https://doi.org/10.1007/s11042-018-6038-x
  12. Tao, X., Zhang, D., Ma, W., Liu, X., and Xu, D., Automatic Metallic Surface Defect Detection and Recognition with Convolutional Neural Networks, Applied Sciences, 2018, Vol. 8, No. 9, p. 1575.
  13. Wang, R., Xie, Y., Yang, J., Xue, L., Hu, M., and Zhang, Q., Large Scale Automatic Image Annotation Based on Convolutional Neural Network, Journal of Visual Communication and Image Representation, 2017, Vol. 49, pp. 213-224. https://doi.org/10.1016/j.jvcir.2017.07.004
  14. Wei, W., Wu, Q., Chen, D., Zhang, Y., Liu, W., Duan, G., and Luo, X., Automatic Image Annotation Based on an Improved Nearest Neighbor Technique with Tag Semantic Extension Model, Procedia Computer Science, 2021, Vol. 183, pp. 616-623. https://doi.org/10.1016/j.procs.2021.02.105
  15. Wei, X., Yang, Z., Liu, Y., Wei, D., Jia, L., and Li, Y., Railway Track Fastener Defect Detection Based on Image Processing and Deep Learning Techniques: A Comparative Study, Engineering Applications of Artificial Intelligence, 2019, Vol. 80, pp. 66-81. https://doi.org/10.1016/j.engappai.2019.01.008