Browse > Article
http://dx.doi.org/10.3745/JIPS.02.0182

Feature Analysis for Detecting Mobile Application Review Generated by AI-Based Language Model  

Lee, Seung-Cheol (Dept. of Computer Engineering, Yeungnam University)
Jang, Yonghun (Dept. of Computer Engineering, Yeungnam University)
Park, Chang-Hyeon (Dept. of Computer Engineering, Yeungnam University)
Seo, Yeong-Seok (Dept. of Computer Engineering, Yeungnam University)
Publication Information
Journal of Information Processing Systems / v.18, no.5, 2022 , pp. 650-664 More about this Journal
Abstract
Mobile applications can be easily downloaded and installed via markets. However, malware and malicious applications containing unwanted advertisements exist in these application markets. Therefore, smartphone users install applications with reference to the application review to avoid such malicious applications. An application review typically comprises contents for evaluation; however, a false review with a specific purpose can be included. Such false reviews are known as fake reviews, and they can be generated using artificial intelligence (AI)-based text-generating models. Recently, AI-based text-generating models have been developed rapidly and demonstrate high-quality generated texts. Herein, we analyze the features of fake reviews generated from Generative Pre-Training-2 (GPT-2), an AI-based text-generating model and create a model to detect those fake reviews. First, we collect a real human-written application review from Kaggle. Subsequently, we identify features of the fake review using natural language processing and statistical analysis. Next, we generate fake review detection models using five types of machine-learning models trained using identified features. In terms of the performances of the fake review detection models, we achieved average F1-scores of 0.738, 0.723, and 0.730 for the fake review, real review, and overall classifications, respectively.
Keywords
Artificial Intelligence; Fake Review; GPT-2; Language Model; Machine Learning; Software Engineering;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Y. Nishi, A. Suge, and H. Takahashi, "Construction of news article evaluation system using language generation model," in Agents and Multi-Agent Systems: Technologies and Applications 2020. Singapore: Springer, 2020, pp. 313-320.
2 Y. Jang, C. H. Park, and Y. S. Seo, "Fake news analysis modeling using quote retweet," Electronics, vol. 8, no, 12, article no. 1377, 2019. https://doi.org/10.3390/electronics8121377   DOI
3 S. Y. Choi, C. G. Lim, and Y. M. Kim, "Automated link tracing for classification of malicious websites in malware distribution networks," Journal of Information Processing Systems, vol. 15, no, 1, pp. 100-115, 2019.   DOI
4 D. He, M. Pan, K. Hong, Y. Cheng, S. Chan, X. Liu, and N. Guizani, "Fake review detection based on PU learning and behavior density," IEEE Network, vol. 34, no. 4, pp. 298-303, 2020.   DOI
5 A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, "Language models are unsupervised multitask learners," 2019 [Online]. Available: https://d4mucfpksywv.cloudfront.net/better-languagemodels/language-models.pdf.
6 R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi, "Defending against neural fake news," Advances in Neural Information Processing Systems, vol. 32, pp. 9054-9065, 2019.
7 J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: pre-training of deep bidirectional transformers for language understanding," 2019 [Online]. Available: https://arxiv.org/abs/1810.04805.
8 L. Zhang, X. Y. Huang, J. Jiang, and Y. K. Hu, "CSLabel: an approach for labelling mobile app reviews," Journal of Computer Science and Technology, vol. 32, no. 6, pp. 1076-1089, 2017.   DOI
9 D. Martens and W. Maalej, "Towards understanding and detecting fake reviews in app stores," Empirical Software Engineering, vol. 24, no. 6, pp. 3316-3355, 2019.   DOI
10 V. Kieuvongngam, B. Tan, and Y. Niu, "Automatic text summarization of COVID-19 medical research articles using BERT and GPT-2," 2020 [Online]. Available: https://arxiv.org/abs/2006.01997.
11 D. I. Adelani, H. Mai, F. Fang, H. H. Nguyen, J. Yamagishi, and I. Echizen, "Generating sentiment-preserving fake online reviews using machine language models and their human- and machine-based detection," in Advanced Information Networking and Applications. Cham, Switzerland: Springer, 2020, pp. 1341-1354.
12 S. Gehrmann, H. Strobelt, and A. M. Rush, "GLTR: statistical detection and visualization of generated text," [2019]. Available: https://arxiv.org/abs/1906.04043.
13 A. Destine-DeFreece, S. Handelsman, T. Light Rake, A. Merkel, and G. Moses, "Can GPT-2 replace a Sex and the City writers' room?," 2019 [Online]. Available: https://digital.kenyon.edu/dh_iphs_ai/15/.
14 Y. Liao, Y. Wang, Q. Liu, and X. Jiang, "GPT-based generation for classical Chinese," 2019 [Online]. Available: https://arxiv.org/abs/1907.00151.
15 K. Ouazzane, J. Li, H. B. Jun, Y. Jing, and R. Boyd, "An artificial Intelligence-based language modeling framework," Expert Systems with Applications, vol. 39, no, 5, pp. 5960-5970, 2012.   DOI
16 S. Barrio, "Writing the next American hit: using GPT-2 to explore the possibility of creating successful AIgenerated song lyrics," 2020 [Online]. Available: https://digital.kenyon.edu/cgi/viewcontent.cgi?article=1011&context=dh_iphs_prog.
17 S. Kreps, R. M. McCain, and M. Brundage, "All the news that's fit to fabricate: AI-generated text as a tool of media misinformation," Journal of Experimental Political Science, vol. 9, no. 1, pp. 104-117, 2022.   DOI
18 T. Fagni, F. Falchi, M. Gambini, A. Martella, and M. Tesconi, "TweepFake: about detecting deepfake tweets," 2021 [Online]. Available: https://arxiv.org/abs/2008.00036.
19 A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, "Improving language understanding by generative pre-training," 2018 [Online]. Available: https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018improving.pdf.
20 W. Huang, X. Liao, Z. Xie, J. Qian, B. Zhuang, S. Wang, and J. Xiao, "Generating reasonable legal text through the combination of language modeling and question answering," in Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI), Virtual Event, 2020, pp. 3687-3693.
21 M. Talal, A. A. Zaidan, B. B. Zaidan, O. S. Albahri, M. A. Alsalem, A. S. Albahri, et al., "Comprehensive review and analysis of anti-malware apps for smartphones," Telecommunication Systems, vol. 72, no, 2, pp. 285-337, 2019.   DOI
22 Y. Liu, Z. Bao, Z. Zhang, D. Tang, and F. Xiong, "Information cascades prediction with attention neural network," Human-centric Computing and Information Sciences, vol. 10, article no, 13, 2020. https://doi.org/10.1186/s13673-020-00218-w   DOI
23 Z. Zhang, J. Jing, X. Wang, K. K. R. Choo, and B. B. Gupta, "A crowdsourcing method for online social networks security assessment based on human-centric computing," Human-centric Computing and Information Sciences, vol. 10, article no, 23, 2020. https://doi.org/10.1186/s13673-020-00230-0   DOI
24 M. Harman, Y. Jia, and Y. Zhang, "App store mining and analysis: MSR for app stores," in Proceedings of 2012 9th IEEE Working Conference on Mining Software Repositories (MSR), Zurich, Switzerland, 2012, pp. 108-111.
25 N. Genc-Nayebi and A. Abran, "A systematic literature review: opinion mining studies from mobile app store user review," Journal of Systems and Software, vol. 125, pp. 207-219, 2017.   DOI
26 Y. S. Jeong and J. H. Park, "Learning algorithms in AI system and services," Journal of Information Processing System, vol. 15, no, 5, pp. 1029-1035, 2019.   DOI
27 A. See, A. Pappu, R. Saxena, A. Yerukola, and C. D. Manning, "Do massively pretrained language models make better storytellers?," 2019 [Online]. Available: https://arxiv.org/abs/1909.10705.
28 M. Ott, Y. Choi, C. Cardie, and J. T. Hancock, "Finding deceptive opinion spam by any stretch of the imagination," in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, 2011, pp. 309-319.
29 H. Chen, D. He, S. Zhu, and J. Yang, "Toward detecting collusive ranking manipulation attackers in mobile app markets," in Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, Abu Dhabi, United Arab Emirates, 2017, pp. 58-70.
30 T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, et al., "Language models are few-shot learners," Advances in Neural Information Processing Systems, vol. 33, pp. 1877-1901, 2020.
31 J. S. Lee and J. Hsiang, "Patent classification by fine-tuning BERT language," World Patent Information, vol. 61, article no. 101965, 2020. https://doi.org/10.1016/j.wpi.2020.101965   DOI
32 J. Salminen, M. Hopf, S. A. Chowdhury, S. G. Jung, H. Almerekhi, and B. J. Jansen, "Developing an online hate classifier for multiple social media platforms," Human-centric Computing and Information Sciences, vol. 10, article no. 1, 2020. https://doi.org/10.1186/s13673-019-0205-6   DOI
33 Z. Horvitz, N. Do, and M. L. Littman, "Context-driven satirical headline generation," in Proceedings of the 2nd Workshop on Figurative Language Processing, Virtual Event, 2020, pp. 40-50.
34 J. Peng, P. Ni, J. Zhu, Z. Dai, Y. Li, G. Li, and X. Bai, "Automatic generation of electronic medical record based on GPT2 model," in Proceedings of 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, 2020, pp. 6180-6182.