Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.2.228

Change Attention-based Vehicle Scratch Detection System  

Lee, EunSeong (Department of Computer Engineering, Kwangwoon Uninversity)
Lee, DongJun (Department of Computer Engineering, Kwangwoon Uninversity)
Park, GunHee (Department of Electronic Engineering, Kwangwoon University)
Lee, Woo-Ju (Department of Computer Engineering, Kwangwoon Uninversity)
Sim, Donggyu (Department of Computer Engineering, Kwangwoon Uninversity)
Oh, Seoung-Jun (Department of Electronic Engineering, Kwangwoon University)
Publication Information
Journal of Broadcast Engineering / v.27, no.2, 2022 , pp. 228-239 More about this Journal
Abstract
In this paper, we propose an unmanned vehicle scratch detection deep learning model for car sharing services. Conventional scratch detection models consist of two steps: 1) a deep learning module for scratch detection of images before and after rental, 2) a manual matching process for finding newly generated scratches. In order to build a fully automatic scratch detection model, we propose a one-step unmanned scratch detection deep learning model. The proposed model is implemented by applying transfer learning and fine-tuning to the deep learning model that detects changes in satellite images. In the proposed car sharing service, specular reflection greatly affects the scratch detection performance since the brightness of the gloss-treated automobile surface is anisotropic and a non-expert user takes a picture with a general camera. In order to reduce detection errors caused by specular reflected light, we propose a preprocessing process for removing specular reflection components. For data taken by mobile phone cameras, the proposed system can provide high matching performance subjectively and objectively. The scores for change detection metrics such as precision, recall, F1, and kappa are 67.90%, 74.56%, 71.08%, and 70.18%, respectively.
Keywords
car sharing; deep learning; change detection; change attention; specularity removal;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Developing Car Damage Detection Model using Semantic Segmentation. (accessed 2020. Feb. 13) doi: https://tech.socarcorp.kr/data/2020/02/13/car-damage-segmentation-model.html
2 H. Chen, and S. Zhenwei, "A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection," Remote Sensing, Vol.12, No.10, pp.1-23, May 2020. doi: https://doi.org/10.3390/rs12101662   DOI
3 C. Stauffer and W. E. L. Grimson, "Learning patterns of activity using real-time tracking," in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, pp. 747-757, Aug 2000. doi: https://doi.org/10.1109/34.868677   DOI
4 D. R. Edgington, K. A. Salamy, M. Risi, R. E. Sherlock, D. Walther and C. Koch, "Automated event detection in underwater video," Oceans 2003. Celebrating the Past ... Teaming Toward the Future (IEEE Cat. No.03CH37492), Vol.5, pp. 2749-2753, 2003. doi: https://doi.org/10.1109/OCEANS.2003.178344   DOI
5 Y. Liu, C. Pang, Z. Zhan, X. Zhang, and X. Yang, "Building Change Detection for Remote Sensing Images Using a Dual Task Constrained Deep Siamese Convolutional Network Model," arXiv preprint, arXiv:1909.07726, 2019. doi: https://doi.org/10.48550/arXiv.1909.07726   DOI
6 L. Khelifi and M. Mignotte, "Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis," IEEE Access, Vol. 8, pp. 126385 -126400. (accessed Jul. 8 2020) doi: https://doi.org/10.1109/ACCESS.2020.3008036   DOI
7 R. C. Daudt, R. L. Saux, and A. Boulch, "Fully convolutional siamese networks for change detection," Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 4063-4067, 2018. (accessed Sep. 6 2018) doi: 10.1109/ICIP.2018.8451652   DOI
8 Gyeonggi Research Institute(GRI), Growing Pains of Car Sharing Services, Beyond Conflicts and Regulations to the Future Gyeonggi Research Institute(GRI) No.440, 2020 Nov. 15. https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE10541172
9 Korea Consumer Agency, Survey Car sharing Terms of Use and Advertisements, 1-79, 2017 May. https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE07456356
10 Q. Zhang, X. Chang and S. Bian Bian, "Vehicle-Damage-Detection Segmentation Algorithm Based on Improved Mask RCNN," IEEE Access, Vol.8, pp.6997-7004, Jan. 06, 2020. doi: https://doi.org/10.1109/ACCESS.2020.2964055   DOI
11 U. Waqas, N. Akram, S. Kim, D. Lee and J. Jeo, "Vehicle Damage Classification and Fraudulent Image Detection Including Moire Effect Using Deep Learning", 2020 IEEE Canadian Conference on Electrical and Computer Engineering, Aug. 30 - Sep. 2 2020. (accessed Nov. 19, 2020) doi: https://doi.org/10.1109/ CCECE47787.2020.9255806   DOI
12 H. S. Malik, M. Dwivedi, S. N. Omakar, S. R. Samal, A. Rathi, E. B. Monis, B. Khanna and A. Tiwari, "Deep Learning Based Car Damage Classification and Detection," Advances in Artificial Intelligence and Data Engineering, Vol.1133, pp 207-221, Aug 2020. https://easychair.org/publications/preprint/RlQ2
13 Y. Zhan, K. Fu, M. Yan, X. Sun, H. Wang and X. Qiu, "Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images," IEEE Geoscience and Remote Sensing Letters, Vol.14, Issue: 10, pp.1845-1849, Oct. 2017. doi: https://doi.org/10.1109/LGRS.2017.2738149   DOI
14 A. Artusi, F. Banterle, and D. Chetverikov, "A Survey of Specularity Removal Methods," COMPUTER GRAPHICS forum, Vol.30, No.8, pp. 2208-2230, 2011. doi: https://doi.org/10.1111/j.1467-8659.2011.01971.x   DOI
15 M. E. A. Larabi, S. Chaib, K. Bakhtj, and M. S. Karoui, "Transfer Learning for Changes Detection in Optical Remote Sensing Imagery," Proceedings of the IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp.1582-1585, 2019. doi: https://doi.org/10.1109/IGARSS.2019.8900296   DOI
16 G. Hwang, W.J. Lee, and S.J. Oh, "Change Attention based Dense Siamese Network for Remote Sensing Change Detection," Journal of Broadcast Engineering, Vol.26, No.1, pp.14-25, Jan 2021. doi: https://doi.org/10.5909/JBE.2021.26.1.14   DOI
17 R. T. Collins, A. J. Lipton and T. Kanade, "Introduction to the special section on video surveillance," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 8, pp. 745-746, Aug 2000. doi: https://doi.org/10.1109/TPAMI.2000.868676   DOI
18 M. Bosc, F. Heitz, J. P. Armspach, I. Namer, D. Gounot and L. Rumbach, "Automatic change detection in multimodal serial MRI: Application to multiple sclerosis lesion evolution," Neuroimage, Vol. 20, pp. 643-656, 2003. doi: https://doi.org/10.1016/s1053-8119(03)00406-3   DOI
19 D. Rey, G. Subsol, H. Delingette and N. Ayache, "Automatic detection and segmentation of evolving processes in 3D medical images: Application to multiple sclerosis," Med. Image Anal, Vol.6, No.2, pp.163-179, Jun 2002. doi: https://doi.org/10.1016/s1361-8415(02)00056-7   DOI
20 K. Lebart, E. Trucco and D. M. Lane, "Real-time automatic sea-floor change detection from video," Oceans Conference Record, Vol. 2, No. 7, pp. 1337-1343, 2000. doi: https://doi.org/10.1109/OCEANS.2000.881789   DOI
21 S. A. Shafer, "Using color to separate reflection components", Color Res. Appl. 10, 210-218, 1985. doi: https://doi.org/10.1002/col.5080100409   DOI
22 K. Yoon, Y. Choi and I.S. Kweon,"Fast Separation of Reflection Components using a Specularity-Invariant Image Representation," 2006 International Conference on Image Processing, Atlanta, Georgia, USA, pp. 973-97, Oct 2006. doi: https://doi.org/10.1109/ICIP.2006.312650   DOI
23 H.L. Shen and Q.Y. Cai, "Simple and efficient method for specularity removal in an image," Virtual Journal for Biomedical Optics, Vol. 4, Iss.7, May. 6 2009. doi: https://doi.org/10.1364/AO.48.002711   DOI
24 X. Xu, J. Lin, Y. Tao and X. Wang, "An Improved DenseNet Method Based on Transfer Learning for Fundus Medical Images," 2018 7th International Conference on Digital Home (ICDH), pp. 137-140, Nov. 30 -Dec. 1 2018. doi: https://doi.org/10.1109/ICDH.2018.00033   DOI
25 R. T. Tan, K. Nishino, and K. Ikeuchi, "Separating reflection components based on chromaticity and noise analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, pp.1373-1379, Oct 2004. doi: https://doi.org/10.1109/TPAMI.2004.90   DOI
26 R. T. Tan and K. Ikeuchi, "Separating reflection components of textured surfaces using a single image," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Iss.2, pp. 178 -193, Feb 2005. doi: https://doi.org/10.1109/TPAMI.2005.36   DOI
27 J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, "Dual attention network for scene segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, California, USA, pp.3146-3154, 2019. doi: https://doi.org/10.48550/arXiv.1809.02983   DOI
28 J. Yosinski, J. Clune, Y. Bengio and H. Lipson, "How transferable are features in deep neural networks?" Advances in Neural Information Processing Systems, Vol.27, pp.3320-3328, Dec. 2014. doi: https://doi.org/10.48550/arXiv.1411.1792   DOI
29 C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang and C. Liu, "A Survey on Deep Transfer Learning," Artificial Neural Networks and Machine Learning, pp. 270-279. (accessed Oct 2018) doi:10.1007/978-3-030-01424-7_27   DOI
30 S. J. Pan and Q. Yang, "A Survey on Transfer Learning," IEEE Transactions on Knowledge and Data Engineering, Vol. 22, Iss.10, pp. 1345-1359, Oct. 2010. doi: https://doi.org/10.1109/TKDE.2009.191   DOI
31 A. Asokan, and J. Anitha, "Change detection techniques for remote sensing applications: a survey." Earth Science Informatics, Vol.12, No.2, pp.143-160, March 2019. doi: https://doi.org/10.1007/s12145-019-00380-5   DOI
32 L. Richard, and G. G. Koch. "The Measurement of Observer Agreement for Categorical Data." Wiley, International Biometric Society, Vol. 33, No. 1, pp. 159-74, 1977. doi: https://doi.org/10.2307/2529310   DOI
33 C. G. P. Suescun, J. O. P. Arenas and R. J. Moreno, "Detection of Scratches on Cars by Means of CNN and R-CNN," International Journal on Advanced Science, Engineering and Information Technology, Vol.9, No.3, pp.745-752, 2019. doi: http://dx.doi.org/10.18517/ijaseit.9.3.6470   DOI
34 F. Wang and D.M.J. Tax. "Survey on the attention based RNN model and its applications in computer vision." arXiv preprint arXiv: 1601.06823, 2016. doi: https://doi.org/10.48550/arXiv.1601.06823   DOI
35 X. Zhu, S. Liu, P. Zhang and Y. Duan, "A Unified Framework of Intelligent Vehicle Damage Assessment based on Computer Vision Technology," 2019 IEEE 2nd International Conference on Automation, Nov. 22-24 2019. doi: https://doi.org/10.1109/AUTEEE48671.2019.9033150   DOI
36 S. K. Nayar, X. S. Fang, and T. Boult, "Separation of reflection components using color and polarization," International Journal of Computer Vision, Vol. 21, pp.163-186, Feb 1997. doi: https://doi.org/10.1023/A:1007937815113   DOI
37 S. Wang, D. Quan, X. Liang, M. Ning, Y. Guo and L. Jiao, "A deep learning framework for remote sensing image registration". ISPRS Journal of Photogrammetry and Remote Sensing, Vol.145, pp.148-164, 2018. doi: http://dx.doi.org/10.1016/j.isprsjprs.2017.12.012   DOI