Browse > Article
http://dx.doi.org/10.9717/kmms.2021.24.9.1224

A Comparative Study on Deepfake Detection using Gray Channel Analysis  

Son, Seok Bin (Dept. of Information Security, College of Interdisciplinary Studies for Emerging Industries, Seoul Women's University)
Jo, Hee Hyeon (Dept. of Information Security, College of Interdisciplinary Studies for Emerging Industries, Seoul Women's University)
Kang, Hee Yoon (Dept. of Information Security, College of Interdisciplinary Studies for Emerging Industries, Seoul Women's University)
Lee, Byung Gul (Dept. of Data Science, College of Interdisciplinary Studies for Emerging Industries, Seoul Women's University)
Lee, Youn Kyu (Dept. of Computer Engineering, College of Engineering, Hongik University)
Publication Information
Journal of Korea Multimedia Society / v.24, no.9, 2021 , pp. 1224-1241 More about this Journal
Abstract
Recent development of deep learning techniques for image generation has led to straightforward generation of sophisticated deepfakes. However, as a result, privacy violations through deepfakes has also became increased. To solve this issue, a number of techniques for deepfake detection have been proposed, which are mainly focused on RGB channel-based analysis. Although existing studies have suggested the effectiveness of other color model-based analysis (i.e., Grayscale), their effectiveness has not been quantitatively validated yet. Thus, in this paper, we compare the effectiveness of Grayscale channel-based analysis with RGB channel-based analysis in deepfake detection. Based on the selected CNN-based models and deepfake datasets, we measured the performance of each color model-based analysis in terms of accuracy and time. The evaluation results confirmed that Grayscale channel-based analysis performs better than RGB-channel analysis in several cases.
Keywords
Deepfake Detection; Grayscale; Gray; Channel; Deep Learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 I. Kassem and A. Sleit, "Elapsed Time of IoT Application Protocol for ECG: A Comparative Study Between CoAP and MQTT," International Conference on Electrical, Communication, and Computer Engineering. IEEE, pp. 1-6, 2020.
2 D. Guera and E.J. Delp, "Deepfake Video Detection Using Recurrent Neural Networks," IEEE International Conference on Advanced Video and Signal Based Surveillance. IEEE, pp. 1-6, 2018.
3 C. Szegedy, S. Ioffe, V. Vanhoucke, and A.A. Alemi, "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31, No. 1, pp. 4278-4284, 2017.
4 J. Kim, S. A. Hong, and H. Kim, "A StyleGAN Image Detection Model Based on Convolutional Neural Network," Journal of Korea Multimedia Society, Vol. 22, No. 12, pp. 1447-1456, 2019.   DOI
5 P. Korshunov and S. Marcel, "Deepfakes: a New Threat to Face Recognition? Assessment and Detection," arXiv preprint, arXiv: 1812.08685, 2018.
6 Y. Li and S. Lyu, "Exposing Deepfake Videos By Detecting Face Warping Artifacts," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 46-52, 2019.
7 L. Jiang, R. Li, W. Wu, C. Qian, and C.C. Loy, "DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2889-2898, 2020.
8 Contributing data to Deepfake detection research(2019). https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html (accessed September 29, 2021).
9 B. Zi, M. Chang, J. Chen, X. Ma, and Y.G. Jiang, "WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection," Proceedings of the ACM International Conference on Multimedia, pp. 2382-2390, 2020.
10 M. Lin, Q. Chen, and S. Yan, "Network In Network," arXiv preprint, arXiv:1312.4400, 2013.
11 S. Narkhede, "Understanding AUC-ROC Curve," Towards Data Science, Vol. 26, pp. 220-227, 2018.
12 Z.C. Lipton, C. Elkan, and B. Naryanaswamy, "Optimal Thresholding of Classifiers to Maximize F1 Measure," Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 225-239, 2014.
13 O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, et al., "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, Vol. 115, No. 3, pp. 211-252, 2015.   DOI
14 A. Pishori, B. Rollins, N. van Houten, N. Chatwani, and O. Uraimov, "Detecting Deep-fake Videos: An Analysis of Three Techniques," arXiv preprint, arXiv:2007.08517, 2020.
15 C.M. Yu, C.T. Chang, and Y.W. Ti, "Detecting Deepfake-Forged Contents with Separable Convolutional Neural Network and Image Segmentation," arXiv preprint, arXiv:1912.12184, 2019.
16 H. Zhou, Y. Liu, Z. Liu, P. Luo, and X. Wang, "Talking Face Generation by Adversarially Disentangled Audio-Visual Representation," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 1, pp. 9299-9306, 2019.
17 U.A. Ciftci, I. Demir, and L. Yin, "FakeCatcher: Detection of Synthetic Portrait Videos Using Biological Signals," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2020.
18 G. Fox, W. Liu, H. Kim, H.P. Seidel, M. Elgharib, and C. Theobalt, "Videoforensicshq: Detecting High-Quality Manipulated Face Videos," IEEE International Conference on Multimedia and Expo. IEEE, pp. 1-6, 2021.
19 Law Commission, Abusive and Offensive Online Communications: A Scoping Report, The Law Commission, London, 2018.
20 G. Huang, Z. Liu, L.V.D. Maaten, and K.Q. Weinberger, "Densely Connected Convolutional Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700-4708, 2017.
21 I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., "Generative Adversarial Nets," arXiv preprint, arXiv:1406.2661, 2014.
22 A. Krizhevsky, I. Sutskever, and G.E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Advances in Neural Information Processing Systems, Vol. 25, pp. 1097-1105, 2012.
23 Y. Mirsky and W. Lee, "The Creation and Detection of Deepfakes: A Survey," ACM Computing Surveys, Vol. 54, No. 1, pp. 1-41, 2021.   DOI
24 D. Chen, Q. Chen, J. Wu, X. Yu, and T. Jia, "Face Swapping: Realistic Image Synthesis Based on Facial Landmarks Alignment," Mathematical Problems in Engineering, pp. 1-11, 2019.
25 F. Mokhayeri, K. Kamali, and E. Granger, "Cross-Domain Face Synthesis using a Controllable GAN," Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 252-260, 2020.
26 Y. Li, Detecting and Protecting against AI-Synthesized Faces, Master's Thesis of State University of New York at Albany, 2020.
27 J. Thies, M. Zollhofer, and M. Niessner, "Deferred Neural Rendering: Image Synthesis Using Neural Textures," ACM Transactions on Graphics, Vol. 38, No. 4, pp. 1-12, 2019.
28 M.S. Rana and A.H. Sung, "DeepfakeStack: A Deep Ensemble-based Learning Technique for Deepfake Detection," IEEE International Conference on Cyber Security and Cloud Computing/IEEE International Conference on Edge Computing and Scalable Cloud. IEEE, pp. 70-75, 2020.
29 B. van Leeuwen, Deepfake Detection Using Convolutional Neural Networks: Working Towards Understanding the Effects of Design Choices, Master's Thesis of Delft University of Technology, 2020.
30 X. Yang, Y. Li, and S. Lyu, "Exposing Deep Fakes Using Inconsistent Head Poses," ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, pp. 8261-8265, 2019.
31 B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, et al., "The DeepFake Detection Challenge (DFDC) Dataset," arXiv preprint, arXiv:1910.08854, 2019.
32 M. Westerlund, "The Emergence of Deepfake Technology: A Review," Technology Innovation Management Review, Vol. 9, No. 11, pp. 39-52, 2019.   DOI
33 Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, "Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3207- 3216, 2020.
34 A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Niessner, "FaceForensics++: Learning to Detect Manipulated Facial Images," Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1-11, 2019.
35 S. McCloskey and M. Albright, "Detecting GAN-generated Imagery using Color Cues," IEEE International Conference on Image Processing, pp. 4584-4588, 2019.
36 I. Ahmad, I. Moon, and S.J. Shin, "Color- to-Grayscale Algorithms Effect on Edge Detection - A Comparative Study," International Conference on Electronics, Information, and Communication. IEEE, pp. 1-4, 2018.
37 Y. Xie and D. Richmond, "Pre-training on Grayscale ImageNet Improves Medical Image Classification," Proceedings of the European Conference on Computer Vision Workshops, pp. 476-484, 2018.
38 F. Chollet, "Xception: Deep Learning With Depthwise Separable Convolutions," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251-1258, 2017.
39 K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
40 K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv preprint, arXiv: 1409.1556, 2015.