지능정보연구 (Journal of Intelligence and Information Systems)

  • 제29권2호
  • /
  • Pages.35-56
  • /
  • 2023
  • /
  • 2288-4866(pISSN)
  • /
  • 2288-4882(eISSN)
한국지능정보시스템학회 (Korea Intelligent Information System Society)
권호 표지
DOI QR코드

DOI QR Code

XAI 기법을 이용한 리뷰 유용성 예측 결과 설명에 관한 연구

Explainable Artificial Intelligence Applied in Deep Learning for Review Helpfulness Prediction

  • 류동엽 (경희대학교 빅데이터 응용학과) ;
  • 이흠철 (경희대학교 빅데이터 응용학과) ;
  • 김재경 (경희대학교 경영대학 & 빅데이터 응용학과)
  • Dongyeop Ryu (Department of Big Data Analysis, Kyung Hee University) ;
  • Xinzhe Li (Department of Big Data Analysis, Kyung Hee University) ;
  • Jaekyeong Kim (School Management & Department of Big Data Analysis, Kyung Hee University)
  • 투고 : 2022.11.10
  • 심사 : 2022.12.14
  • 발행 : 2023.06.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

정보통신 기술의 발전에 따라 웹 사이트에는 수많은 리뷰가 지속적으로 게시되고 있다. 이로 인해 정보 과부하 문제가 발생하여 사용자들은 본인이 원하는 리뷰를 탐색하는데 어려움을 겪고 있다. 따라서, 이러한 문제를 해결하여 사용자에게 유용하고 신뢰성 있는 리뷰를 제공하기 위해 리뷰 유용성 예측에 관한 연구가 활발히 진행되고 있다. 기존 연구는 주로 리뷰에 포함된 특성을 기반으로 리뷰 유용성을 예측하였다. 그러나, 예측한 리뷰가 왜 유용한지 근거를 제시할 수 없다는 한계점이 존재한다. 따라서 본 연구는 이러한 한계점을 해결하기 위해 리뷰 유용성 예측 모델에 eXplainable Artificial Intelligence(XAI) 기법을 적용하는 방법론을 제안하였다. 본 연구는 Yelp.com에서 수집한 레스토랑 리뷰를 사용하여 리뷰 유용성 예측에 관한 연구에서 널리 사용되는 6개의 모델을 통해 예측 성능을 비교하였다. 그 다음, 예측 성능이 가장 우수한 모델에 XAI 기법을 적용하여 설명 가능한 리뷰 유용성 예측 모델을 제안하였다. 따라서 본 연구에서 제안한 방법론은 사용자의 구매 의사결정 과정에서 유용한 리뷰를 추천할 수 있는 동시에 해당 리뷰가 왜 유용한지에 대한 해석을 제공할 수 있다.

With the development of information and communication technology, numerous reviews are continuously posted on websites, which causes information overload problems. Therefore, users face difficulty in exploring reviews for their decision-making. To solve such a problem, many studies on review helpfulness prediction have been actively conducted to provide users with helpful and reliable reviews. Existing studies predict review helpfulness mainly based on the features included in the review. However, such studies disable providing the reason why predicted reviews are helpful. Therefore, this study aims to propose a methodology for applying eXplainable Artificial Intelligence (XAI) techniques in review helpfulness prediction to address such a limitation. This study uses restaurant reviews collected from Yelp.com to compare the prediction performance of six models widely used in previous studies. Next, we propose an explainable review helpfulness prediction model by applying the XAI technique to the model with the best prediction performance. Therefore, the methodology proposed in this study can recommend helpful reviews in the user's purchasing decision-making process and provide the interpretation of why such predicted reviews are helpful.

키워드

과제정보

본 논문은 교육부 및 한국연구재단 4단계 두뇌한국21 사업(4단계 BK21 사업)으로부터 지원받은 연구임.

참고문헌

  1. 이청용, 이병현, 이흠철, & 김재경. (2021). CNN 기반 리뷰 유용성 점수 예측을 통한 개인화 추천 서비스 성능 향상에 관한 연구. 지능 정보연구, 27(3), 29-56.
  2. 이흠철, 윤효림, 이청용, & 김재경. (2022). Multichannel CNN 기반 온라인 리뷰 유용성 예측 모델 개발에 관한 연구. 지능정보연구, 28(2), 171-189. https://doi.org/10.13088/JIIS.2022.28.2.171
  3. 이병현, 최일영 정재호, & 김재경. (2022). E-커머스 사용자의 평점과 리뷰 유용성이 상품추천 시스템의 성능 향상에 미치는 영향 분석. 지능정보연구, 28(1), 311-328. https://doi.org/10.13088/JIIS.2022.28.1.311
  4. 박호연, & 김경재.(2019). CNN-LSTM 조합모델을 이용한 영화리뷰 감성분석. 지능정보연구, 25(4), 141-154. https://doi.org/10.13088/JIIS.2019.25.4.141
  5. 주명길, & 윤성욱. (2019). 워드 임베딩과 CNN을 사용하여 영화 리뷰에 대한 감성 분석. 디지털산업정보학회논문지, 15(1), 87-97. https://doi.org/10.17662/KSDIM.2019.15.1.087
  6. Adak, A., Pradhan, B., Shukla, N., & Alamri, A. (2022). Unboxing deep learning model of food delivery service reviews using explainable artificial intelligence (XAI) technique. Foods, 11(14), 2019.
  7. Ariyasriwatana, W., Buente, W., Oshiro, M., & Streveler, D. (2014). Categorizing health-related cues to action: using Yelp reviews of restaurants in Hawaii. New Review of Hypermedia and Multimedia, 20(4), 317-340. https://doi.org/10.1080/13614568.2014.987326
  8. Ariyasriwatana, W., & Quiroga, L. M. (2016). A thousand ways to say 'Delicious!'-Categorizing expressions of deliciousness from restaurant reviews on the social network site Yelp. Appetite, 104, 18-32. https://doi.org/10.1016/j.appet.2016.01.002
  9. Arrieta, A. B., Diaz-Rodriguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., & Benjamins, R. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion, 58, 82-115.
  10. Aslam, N., Khan, I. U., Mirza, S., AlOwayed, A., Anis, F. M., Aljuaid, R. M., & Baageel, R. (2022). Interpretable Machine Learning Models for Malicious Domains Detection Using Explainable Artificial Intelligence (XAI). Sustainability, 14(12), 7375.
  11. Behera, R. K., Jena, M., Rath, S. K., & Misra, S. (2021). Co-LSTM: Convolutional LSTM model for sentiment analysis in social big data. Information Processing & Management, 58(1), 102435.
  12. Bilal, M., & Almazroi, A. A. (2022). Effectiveness of Fine-Tuned BERT Model in Classification of Helpful and Unhelpful Online Customer Reviews. Electronic Commerce Research, 1-21.
  13. Cantallops, A. S., & Salvi, F. (2014). New consumer behavior: A review of research on eWOM and hotels. International Journal of Hospitality Management, 36, 41-51.
  14. Chan, J. Y.-L., Leow, S. M. H., Bea, K. T., Cheng, W. K., Phoong, S. W., Hong, Z.-W., & Chen, Y.-L. (2022). Mitigating the Multicollinearity Problem and Its Machine Learning Approach: A Review. Mathematics, 10(8), 1283.
  15. Chen, H., Han, F. X., Niu, D., Liu, D., Lai, K., Wu, C., & Xu, Y. (2018). Mix: Multichannel information crossing for text matching. Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining,
  16. Chen, W.-K., Riantama, D., & Chen, L.-S. (2020). Using a text mining approach to hear voices of customers from social media toward the fast-food restaurant industry. Sustainability, 13(1), 268.
  17. Choi, H. S., & Leon, S. (2020). An empirical investigation of online review helpfulness: A big data perspective. Decision Support Systems, 139, 113403.
  18. Chua, A. Y., & Banerjee, S. (2016). Helpfulness of user-generated reviews as a function of review sentiment, product type and information quality. Computers in Human Behavior, 54, 547-554. https://doi.org/10.1016/j.chb.2015.08.057
  19. Dikshit, A., & Pradhan, B. (2021). Interpretable and explainable AI (XAI) model for spatial drought prediction. Science of the Total Environment, 801, 149797.
  20. Fader, P. S., & Winer, R. S. (2012). Introduction to the special issue on the emergence and impact of user-generated content. Marketing Science, 31(3), 369-371.
  21. Fang, X., & Zhan, J. (2015). Sentiment analysis using product review data. Journal of Big Data, 2(1), 1-14.
  22. Fiok, K., Karwowski, W., Gutierrez, E., & Wilamowski, M. (2021). Analysis of sentiment in tweets addressed to a single domain-specific Twitter account: Comparison of model performance and explainability of predictions. Expert Systems with Applications, 186, 115771.
  23. Fullerton, L. (2017). Online reviews impact purchasing decisions for over 93% of consumers, report suggests. The Drum.
  24. Handelman, G. S., Kok, H. K., Chandra, R. V., Razavi, A. H., Huang, S., Brooks, M., Lee, M. J., & Asadi, H. (2019). Peering into the black box of artificial intelligence: evaluation metrics of machine learning methods. American Journal of Roentgenology, 212(1), 38-43. https://doi.org/10.2214/AJR.18.20224
  25. Ide, H., & Kurita, T.(2017). Improvement of learning CNN with ReLU activation by sparse regularization. 2017 International Join Conference on Neural Networks(IJCNN), Anchorage, AK, USA
  26. Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996). Artificial neural networks: A tutorial. Computer, 29(3), 31-44.
  27. Jain, D. K., Rahate, A., Joshi, G., Walambe, R., & Kotecha, K. (2022). Employing Co-Learning to Evaluate the Explainability of Multimodal Sentiment Analysis. IEEE Transactions on Computational Social Systems.
  28. Jeyakumar, J. V., Noor, J., Cheng, Y.-H., Garcia, L., & Srivastava, M. (2020). How can i explain this to you? an empirical study of deep neural network explanation methods. Advances in Neural Information Processing Systems, 33, 4211-4222.
  29. Jia, S. (2021). Analyzing restaurant customers' evolution of dining patterns and satisfaction during COVID-19 for sustainable business insights. Sustainability, 13(9), 4981.
  30. Jia, S. S. (2020). Motivation and satisfaction of Chinese and US tourists in restaurants: A cross-cultural text mining of online reviews. Tourism Management, 78, 104071.
  31. Jones, Q., Ravid, G., & Rafaeli, S. (2004). Information overload and the message dynamics of online interaction spaces: A theoretical model and empirical exploration. Information systems research, 15(2), 194-210.
  32. Kang, H., Yoo, S. J., & Han, D. (2012). Senti-lexicon and improved Naive Bayes algorithms for sentiment analysis of restaurant reviews. Expert Systems with Applications, 39(5), 6000-6010. https://doi.org/10.1016/j.eswa.2011.11.107
  33. Kaushik, K., Mishra, R., Rana, N. P., & Dwivedi, Y. K. (2018). Exploring reviews and review sequences on e-commerce platform: A study of helpful reviews on Amazon. in. Journal of retailing and Consumer Services, 45, 21-32. https://doi.org/10.1016/j.jretconser.2018.08.002
  34. Krishnamoorthy, S. (2015). Linguistic features for review helpfulness prediction. Expert Systems with Applications, 42(7), 3751-3759. https://doi.org/10.1016/j.eswa.2014.12.044
  35. Kumar, A., Dikshit, S., & Albuquerque, V. H. C. (2021). Explainable artificial intelligence for sarcasm detection in dialogues. Wireless Communications and Mobile Computing, 2021.
  36. Kwon, W., Lee, M., & Back, K.-J. (2020). Exploring the underlying factors of customer value in restaurants: A machine learning approach. International Journal of Hospitality Management, 91, 102643.
  37. Lee, M., Kwon, W., & Back, K.-J. (2021). Artificial intelligence for hospitality big data analytics: developing a prediction model of restaurant review helpfulness for customer decision-making. International Journal of Contemporary Hospitality Management.
  38. Lee, P.-J., Hu, Y.-H., & Lu, K.-T. (2018). Assessing the helpfulness of online hotel reviews: A classification-based approach. Telematics and Informatics, 35(2), 436-445. https://doi.org/10.1016/j.tele.2018.01.001
  39. Li, Z. (2022). Extracting spatial effects from machine learning model using local interpretation method: An example of SHAP and XGBoost. Computers, Environment and Urban Systems, 96, 101845.
  40. Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020). Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1), 18.
  41. Liu, J., Yu, Y., Mehraliyev, F., Hu, S., & Chen, J. (2022). What affects the online ratings of restaurant consumers: a research perspective on text-mining big data analysis. International Journal of Contemporary Hospitality Management (ahead-of-print).
  42. Lopez, A., & Garza, R. (2021). Do sensory reviews make more sense? The mediation of objective perception in online review helpfulness. Journal of Research in Interactive Marketing.
  43. Lu, K., & Wu, J. (2019). Sentiment analysis of film review texts based on sentiment dictionary and SVM. Proceedings of the 2019 3rd international conference on innovation in artificial intelligence.
  44. Luo, Y., & Xu, X. (2019). Predicting the helpfulness of online restaurant reviews using different machine learning algorithms: A case study of yelp. Sustainability, 11(19), 5254.
  45. Luo, Y., & Xu, X. (2021). Comparative study of deep learning models for analyzing online restaurant reviews in the era of the COVID-19 pandemic. International Journal of Hospitality Management, 94, 102849.
  46. Ma, Y., Xiang, Z., Du, Q., & Fan, W. (2018). Effects of user-provided photos on hotel review helpfulness: An analytical approach with deep leaning. International Journal of Hospitality Management, 71, 120-131.
  47. Maks, I., & Vossen, P. (2012). A lexicon model for deep sentiment analysis and opinion mining applications. Decision Support Systems, 53(4), 680-688. https://doi.org/10.1016/j.dss.2012.05.025
  48. Malik, M., & Hussain, A. (2018). An analysis of review content and reviewer variables that contribute to review helpfulness. Information Processing & Management, 54(1), 88-104.
  49. Malik, M. S. I. (2020). Predicting users' review helpfulness: the role of significant review and reviewer characteristics. Soft Computing, 24(18), 13913-13928. https://doi.org/10.1007/s00500-020-04767-1
  50. Mitra, S., & Jenamani, M. (2021). Helpfulness of online consumer reviews: A multi-perspective approach. Information Processing & Management, 58(3), 102538.
  51. Muhamedyev, R., Yakunin, K., Kuchin, Y., Symagulov, A., Buldybayev, T., Murzakhmetov, S., & Abdurazakov, A. (2020). The use of machine learning "black boxes" explanation systems to improve the quality of school education. Cogent Engineering, 7(1), 1769349.
  52. Musto, C., Lops, P., de Gemmis, M., & Semeraro, G. (2021). Context-aware graph-based recommendations exploiting Personalized PageRank. Knowledge-Based Systems, 216, 106806.
  53. Ngo-Ye, T. L., & Sinha, A. P. (2014). The influence of reviewer engagement characteristics on online review helpfulness: A text regression model. Decision Support Systems, 61, 47-58. https://doi.org/10.1016/j.dss.2014.01.011
  54. Olmedilla, M., Martinez-Torres, M. R., & Toral, S. (2022). Prediction and modelling online reviews helpfulness using 1D Convolutional Neural Networks. Expert Systems with Applications, 198, 116787.
  55. Park, E., Chae, B., Kwon, J., & Kim, W.-H. (2020). The effects of green restaurant attributes on customer satisfaction using the structural topic model on online customer reviews. Sustainability, 12(7), 2843.
  56. Qazi, A., Syed, K. B. S., Raj, R. G., Cambria, E., Tahir, M., & Alghazzawi, D. (2016). A concept-level approach to the analysis of online review helpfulness. Computers in Human Behavior, 58, 75-81. https://doi.org/10.1016/j.chb.2015.12.028
  57. Rai, A. (2020). Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science, 48(1), 137-141. https://doi.org/10.1007/s11747-019-00710-5
  58. Ren, Y., & Ji, D. (2017). Neural networks for deceptive opinion spam detection: An empirical study. Information Sciences, 385, 213-224. https://doi.org/10.1016/j.ins.2017.01.015
  59. Sadiq, S., Umer, M., Ullah, S., Mirjalili, S., Rupapara, V., & Nappi, M. (2021). Discrepancy detection between actual user reviews and numeric ratings of Google App store using deep learning. Expert Systems with Applications, 181, 115111.
  60. Sak, H., Senior, A., & Beaufays, F. (2014). Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128.
  61. Salehan, M., & Kim, D. J. (2016). Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics. Decision Support Systems, 81, 30-40. https://doi.org/10.1016/j.dss.2015.10.006
  62. Saumya, S., Singh, J. P., & Dwivedi, Y. K. (2020). Predicting the helpfulness score of online reviews using convolutional neural network. Soft Computing, 24(15), 10989-11005. https://doi.org/10.1007/s00500-019-03851-5
  63. Sherstinsky, A. (2020). Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena, 404, 132306.
  64. Singh, H., Roy, A., Setia, R., & Pateriya, B. (2022). Estimation of nitrogen content in wheat from proximal hyperspectral data using machine learning and explainable artificial intelligence (XAI) approach. Modeling Earth Systems and Environment, 8(2), 2505-2511. https://doi.org/10.1007/s40808-021-01243-z
  65. Singh, J. P., Irani, S., Rana, N. P., Dwivedi, Y. K., Saumya, S., & Roy, P. K. (2017). Predicting the "helpfulness" of online consumer reviews. Journal of Business Research, 70, 346-355. https://doi.org/10.1016/j.jbusres.2016.08.008
  66. Strumbelj, E., & Kononenko, I. (2014). Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41(3), 647-665. https://doi.org/10.1007/s10115-013-0679-x
  67. Sun, X., Han, M., & Feng, J. (2019). Helpfulness of online reviews: Examining review informativeness and classification thresholds by search products and experience products. Decision Support Systems, 124, 113099.
  68. Vapnik, V. (1999). The nature of statistical learning theory. Springer science & business media.
  69. Wang, J., Yu, L.-C., Lai, K. R., & Zhang, X. (2016). Dimensional sentiment analysis using a regional CNN-LSTM model. Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers),
  70. Wang, W., Wang, H., & Song, Y. (2017). Ranking product aspects through sentiment analysis of online reviews. Journal of Experimental & Theoretical Artificial Intelligence, 29(2), 227-246. https://doi.org/10.1080/0952813X.2015.1132270
  71. Yang, S., Yao, J., & Qazi, A. (2020). Does the review deserve more helpfulness when its title resembles the content? Locating helpful reviews by text mining. Information Processing & Management, 57(2), 102179.
  72. Yin, D., Bond, S. D., & Zhang, H. (2014). Anxious or angry? Effects of discrete emotions on the perceived helpfulness of online reviews. MIS quarterly, 38(2), 539-560.
  73. Zhang, W., & Gao, F. (2011). An improvement to naive bayes for text classification. Procedia Engineering, 15, 2160-2164.
  74. Zhang, Y., & Lin, Z. (2018). Predicting the helpfulness of online product reviews: A multilingual approach. Electronic Commerce Research and Applications, 27, 1-10. https://doi.org/10.1016/j.elerap.2017.10.008
  75. Zhou, C., Yang, S., Chen, Y., Zhou, S., Li, Y., & Qazi, A. (2022). How does topic consistency affect online review helpfulness? The role of review emotional intensity. Electronic Commerce Research, 1-36.
  76. Zhu, L., Yin, G., & He, W. (2014). Is this opinion leader's review useful? Peripheral cues for online review helpfulness. Journal of Electronic Commerce Research, 15(4), 267.