Browse > Article
http://dx.doi.org/10.33778/kcsa.2022.22.5.011

Multi-type object detection-based de-identification technique for personal information protection  

Ye-Seul Kil (성신여자대학교 미래융합기술공학과)
Hyo-Jin Lee (성신여자대학교 미래융합기술공학과)
Jung-Hwa Ryu (성신여자대학교 융합보안공학과)
Il-Gu Lee (성신여자대학교 미래융합기술공학과)
Publication Information
Convergence Security Journal / v.22, no.5, 2022 , pp. 11-20 More about this Journal
Abstract
As the Internet and web technology develop around mobile devices, image data contains various types of sensitive information such as people, text, and space. In addition to these characteristics, as the use of SNS increases, the amount of damage caused by exposure and abuse of personal information online is increasing. However, research on de-identification technology based on multi-type object detection for personal information protection is insufficient. Therefore, this paper proposes an artificial intelligence model that detects and de-identifies multiple types of objects using existing single-type object detection models in parallel. Through cutmix, an image in which person and text objects exist together are created and composed of training data, and detection and de-identification of objects with different characteristics of person and text was performed. The proposed model achieves a precision of 0.724 and mAP@.5 of 0.745 when two objects are present at the same time. In addition, after de-identification, mAP@.5 was 0.224 for all objects, showing a decrease of 0.4 or more.
Keywords
Image Privacy; De-Identification; Artificial Intelligence; Multi Object Detection; Cutmix;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 A. Singh, G. Pang, M. Toh, J. Huang, W. Galuba and T. Hassner, "TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text.", Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
2 TY. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, L. Zitnick and P. Dollar "Microsoft coco: Common objects in context.", European conference on computer vision, Springer, Cham, 2014.
3 A. Veit, T. Matera, L. Nemann, J. Matas and S. Belongie, "Coco-text: Dataset and benchmark for text detection and recognition in natural images.", arXiv:1601.07140, 2016.
4 Zhu, Mu. "Recall, precision and average precision." Department of Statistics and Actuarial Science, University of Waterloo, Waterloo 2.30 (2004): 6.
5 Jain, A.K., Sahoo, S.R. & Kaubiyal, J. "Online social networks security and privacy: comprehensive review and analysis". Complex Intell. Syst. 7, 2157-2177 (2021)   DOI
6 M. Carranza-Garcia , J. Torres-Mateo, P. Lara-Benitez and J. Garcia-Gutierrez, "On the Performance of One-Stage and Two-Stage Object Detectors in Autonomous Vehicles Using Camera Data", Remote Sens. 2021, vol. 13, no. 1, pp. 89, 2020.
7 L. Jiao, F. Zhang, F. Liu, S. Yang, L. Li, Z. Feng and R. Qu., "A Survey of Deep Learning-based Object Detection", arXiv:1907.09408v2, 2019.   DOI
8 H. Chen, Z. He, B. Shi, and T. Zhong, "Research on Recognition Method of Electrical Components Based on YOLO V3", IEEE Access, vol. 7, pp. 157818 - 157829, 2019.
9 J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection", arXiv:1506.02640v5, 2015.
10 J. Redmon and A. Farhadi, "YOLOv3: An Incremental Improvement", arXiv:1804.02767, 2018.
11 Y. Du, C. Li, R. Guo, X. Yin, W. Liu, J. Zhou, Y. Bai, Z. Yu, Y. Yang, Q. Dang and H. Wang, "PP-OCR: A Practical Ultra Lightweight OCR System", arXiv:2009.09941, 2020.
12 Bhujbal, Avinash and Deepak T. Mane. "A Survey On Deep Learning Approaches For Vehicle And Number Plate Detection." International Journal of Scientific & Technology Research 8 (2019): 1378-1383.
13 Y. Baek, B. Lee, D. Han, S. Yun, and H. Lee, "Character Region Awareness for Text Detection", arXiv:1904.01941v1, 2019.
14 S. Yun, D. Han, S. Oh, S. Chun, J. Choe and Y. Yoo, "CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features", arXiv:1905.04899, 2019.
15 SR. Klomp, M. Rijn, R. Wijnhoven, C. Snoek and P. De With, "Safe Fakes: Evaluating Face Anonymizers for Face Detectors", Proceedings of 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021.
16 H. Hukkelas, R. Mester. and F. Lindseth, "DeepPrivacy: A Generative Adversarial Network for Face Anonymization", arXiv:1909.04538, 2019.
17 C. Liu, T. Zhu, J. Zhang and W. Zhou, "Privacy Intelligence: A Survey on Image Privacy in Online Social Networks", arXiv:2008.12199v2, 2020.
18 L. Schnabel, S. Matzka, M. Stellmacher, M. Patzold and E. Matthes, "Impact of Anonymization on Vehicle Detector Performance" Proceedings of 2019 Second International Conference on Artificial Intelligence for Industries (AI4I), 2019.
19 L. Du. and H. Ling., "Preservative License Plate De-identification for Privacy Protection" Proceedings of 2011 International Conference on Document Analysis and Recognition, 2011.
20 J. Yu, H. Xue, B. Liu, Y. Wang, S. Zhu and M. Ding "GAN_based differential private image privacy protection framework" Sensors 2021, vol. 21, no. 1, pp. 58, 2020.
21 Z. Ge, S. Liu, F. Wang, Z. Li and J. Sun, "YOLOX: Exceeding YOLO Series in 2021", arXiv:2107.08430, 2021.
22 N. J. Karthika, and S. Chandran, "Addressing the False Positives in Pedestrian Detection.", Electronic Systems and Intelligent Computing, Springer, Singapore, 2020. 1083-1092.
23 Nanonets, "How to OCR with Tesseract, OpenCV and Python". Nanonets, July 2022, https://nanonets.com/blog/ocr-with-tesseract/, accessed 30 Sep 2022.
24 L. Wang, J. Shi and G. Song& I. Shen, "Object detection combining recognition and segmentation.", Asian conference on computer vision, Springer, Berlin, Heidelberg, 2007.