Browse > Article
http://dx.doi.org/10.9716/KITS.2017.16.4.161

Predicting Missing Ratings of Each Evaluation Criteria for Hotel by Analyzing User Reviews  

Lee, Donghoon (국민대학교 비즈니스IT전문대학원)
Boo, Hyunkyung (국민대학교 비즈니스IT전문대학원)
Kim, Namgyu (국민대학교 경영정보학부)
Publication Information
Journal of Information Technology Services / v.16, no.4, 2017 , pp. 161-176 More about this Journal
Abstract
Recently, most of the users can easily get access to a variety of information sources about companies, products, and services through online channels. Therefore, the online user evaluations are becoming the most powerful tool to generate word of mouth. The user's evaluation is provided in two forms, quantitative rating and review text. The rating is then divided into an overall rating and a detailed rating according to various evaluation criteria. However, since it is a burden for the reviewer to complete all required ratings for each evaluation criteria, so most of the sites requested only mandatory inputs for overall rating and optional inputs for other evaluation criteria. In fact, many users input only the ratings for some of the evaluation criteria and the percentage of missed ratings for each criteria is about 40%. As these missed ratings are the missing values in each criteria, the simple average calculation by ignoring the average 40% of the missed ratings can sufficiently distort the actual phenomenon. Therefore, in this study, we propose a methodology to predict the rating for the missed values of each criteria by analyzing user's evaluation information included the overall rating and text review for each criteria. The experiments were conducted on 207,968 evaluations collected from the actual hotel evaluation site. As a result, it was confirmed that the prediction accuracy of the detailed criteria ratings by the proposed methodology was much higher than the existing average-based method.
Keywords
Big Data; Review Analysis; Text Mining; Topic Modeling;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Moghaddam, S. and E. Martin, "ILDA : Interdependent LDA Model for Learning Latent Aspects and Their Ratings from Online Product Reviews", Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2011, 665-674.
2 Mudambi, S.M. and D. Schuff, "What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.com", Journal MIS Quarterly Archive, Vol.34, No.1, 2010, 185-200.   DOI
3 Oh, B.H., J.H. Yang, and H.J. Lee, "A Hybrid Recommender System based on Collaborative Filtering with Selective Utilization of Content-based Predicted Ratings", Journal of KISS : Software and Applications, Vol.41, No.4, 2014, 289-294. (오병화, 양지훈, 이현진, "콘텐츠 기반 예측 평점을 선택적으로 활용하는 협력적 필터링 기반 복합추천 시스템", 정보과학회논문지 : 소프트웨어 및 응용, 제41권, 제4호, 2014, 289-294.)
4 Park, D.H., J. Lee, and I. Han, "The Effect of Online Consumer Reviews on Consumer Purchasing Intention : The Moderating Role of Involvement", International Journal of Electronic Commerce, Vol.11, No.4, 2007, 125-148.   DOI
5 Patrali, C., "Online Reviews: Do Consumers Use Them?", Advances in Consumer Research, Vol.28, No.1, 2001, 129-133.
6 Persson, K., "Predicting Movie Ratings : a Comparative Study on Random Forests and Support Vector Machines", Dissertation, 2015.
7 Seo, S.Y., J. Huangy, H. Yangy, and Y. Liu, "Representation Learning of Users, and Items for Review Rating Prediction Using Attention- based Convolutional Neural Network", 3rd International Workshop on Machine Learning Methods for Recommender Systems, Houston, Texas, 2017.
8 Sharma, R.D., S. Tripathi, S.K. Sahu, S. Mittal, and A. Anand, "Predicting Online Doctor Ratings from User Reviews Using Convolutional Neural Networks", International Journal of Machine Learning, and Computing, Vol.6, No.2, 2016, 149-154. doi : 10.18178/ijmlc.2016.6.2.590 (Downloaded August 30, 2017).   DOI
9 Son, S.B. and J.H. Chun, "Product Feature Extraction and Rating Distribution Using User Reviews", Journal of Society for e-Business Studies, Vol.22, No.1, 2017, 65-87. (손수빈, 전종훈, "사용자 리뷰를 이용한 상품 특징 추출 및 평점 분배", 한국전자거래학회지, 제22권, 제1호, 2017, 65-87.)   DOI
10 Song, C.H., B.C. Kim, and K.D. Park, "Impact of Consumer Rating on Mobile Application Sales : A Comparison between Korean, and American Consumers", Korean Management Review, Vol.43, No.5, 2014, 1493-1518. (송치훈, 김병조, 박경도, "소비자 평점이 모바일 애플리케이션 구매에 미치는 영향-한국과 미국 소비자와의 비교를 중심으로", 경영학연구, 제43권, 제5호, 2014, 1493-1518.)
11 Tang, D., B. Qiny, T. Liuy, and Y. Yang, "User Modeling with Neural Network for Review Rating Prediction", Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.
12 Wang, B., Y. Huang, and X. Li, "Combining Review Text Content, and Reviewer-Item Rating Matrix to Predict Review Rating", Computational Intelligence, and Neuroscience, 2016. doi : 10.1155/2016/5968705 (Downloaded August 29, 2017).   DOI
13 Wang, H., Y. Lu, and C.X. Zhai, "Latent Aspect Rating Analysis without Aspect Keyword Supervision", Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery, and Data Mining (KDD'11), 2011, 618-626.
14 Ye, Q., R. Law, and B. Gu, "The Impact of Online User Reviews on Hotel Room Sales", International Journal of Hospitality Management, Vol.28, No.1, 2009, 180-182. doi : http://dx.doi.org/10.1016/j.ijhm.2015.05.005 (Downloaded August 30, 2017).   DOI
15 Yew, J.L. and W.T. Yee, "Variational Bayesian Approach to Movie Rating Prediction", Association for Computing Machinery, 2007.
16 Yoon, D.M. and K.J. Kim, "Prediction of Rating Score from Short Comments on Movies Using Word-Rating Correlation Analysis", Proceedings of the HCI KOREA, 2011, 484-486. (윤두밈, 김경중, "단어-평점 상관관계 분석을 통한 단문 영화평 평점 예측", 한국HCI학회 학술대회, 2011, 484-486.)
17 Blei, D.M., "Probabilistic Topic Models", Communications of the ACM, Vol.55, No.4, 2012, 77-84. doi : 10.1145/2133806.2133826 (Downloaded August 30, 2017).   DOI
18 Augustine, A. and M. Pathak, "User Rating Prediction for Movies", University of Texas at Austin, 2007.
19 Aurelio G.M. and R. Minazzi, "Web Reviews Influence on Expectations and Purchasing Intentions of Hotel Potential Customers", International Journal of Hospitality Management, Vol.34, 2013, 99-107. doi : http://dx.doi.org/10.1016/j.ijhm.2013.02.012 (Downloaded August 30, 2017).   DOI
20 Bao, Z. and H. Xia, "Movie Rating Estimation and Recommendation", CS229 2012 Project, Stanford University, 2012.
21 Blei, D.M. and J.D. Lafferty, "Dynamic Topic Models", ICML '06 Proceedings of the 23rd International Conference on Machine Learning, 2006, 113-120.
22 Blei, D.M., A.Y. Ng, and M.I. Jordan, "Latent Dirichlet Allocation", Journal of Machine Learning Research, Vol.3, 2003, 993-1022.
23 Casalo, L.V., C. Flavian, M. Guinaliu, and Y. Ekinci, "Do Online Hotel Rating Schemes Influence Booking Behaviors?", International Journal of Hospitality Management, Vol. 49, 2015, 28-36.   DOI
24 Cui, G., H.K. Lui, and X.N. Guo, "The Effect of Online Consumer Reviews on New Product Sales", International Journal of Electronic Commerce, Vol.17, No.1, 2012, 39-58.   DOI
25 Davis, A. and D. Khazanchi, "The Influence of Online Word of Mouth on Product Sales in Retail E-commerce : An Empirical Investigation", Proceedings of the 13th Annual Americas Conference on Information Systems, Vol.13, No.3, 2007, 2149-2157.
26 Duan, W.J., G. Gu, and A.B. Whinston, "Do Online Reviews Matter?-An Empirical Investigation of Panel Data", Decision Support Systems, Vol.45, No.4, 2008, 1007-1016.   DOI
27 Gauri, D.K., A. Bhatnagar, and R. Rao, "Role of Word of Mouth in Online Store Loyalty", Magazine Communications of the ACM, Vol. 51, No.3, 2008, 89-91.
28 Gayatree, G., N. Elhadad, and A. Marian, "Beyond the Stars : Improving Rating Predictions Using Review Text Content", Twelfth International Workshop on the Web and Databases, 2009.
29 Ghose, A. and G.I. Panagiotis, "Designing Novel Review Ranking Systems : Predicting the Usefulness and Impact of Reviews", Proceedings of the Ninth International Conference on Electronic Commerce, 2007, 303-310.
30 Ghose, A. and G.I. Panagiotis, "Designing Ranking Systems for Consumer Reviews : The Impact of Review Subjectivity on Product Sales and Review Quality", Proceedings of the International Conference on Decision Support Systems, 2006.
31 Gupta, N., G.D. Fabbrizio, and P. Haffner, "Capturing the Stars : Predicting Ratings for Service and Product Reviews", Proceedings of the NAACL HLT 2010 Workshop on Semantic Search, Los Angeles, California, 2010, 36-43.
32 Ha, H.S., H.J. Kim, S.W. Choi, and S.R. Yoon, "Comparing and Predicting Movie Rating Using Social Network Service", Proceedings of Korean Institute of Information Scientists and Engineers, 2014, 276-278. (하헌석, 김한주, 최성운, 윤성로, "소셜 네트워크 서비스를 활용한 영화 평점 비교 및 예측", 한국정보과학회 학술발표논문집, 2014, 276-278.)
33 Hofmann, T., "Probabilistic Latent Semantic Indexing", Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, California, 1999, 50-57. doi : 10.1145/312624.312649 (Downloaded August 30, 2017).   DOI
34 Hu, N., N.S. Koh, and S.K. Reddy, "Ratings Lead You to the Product Reviews Help You Clinch It? The Mediating Role of Online Review Sentiments on Product Sales", Decision Support Systems, Vol.57, 2014, 42-53.   DOI
35 Kim, G.M., M.H. Ahn, and Y.H. Lee, "Prediction of Movies based on Naver DB Information and Filtering Malicious Rating Markers", Proceedings of Korean Institute of Industrial Engineers, 2013, 1651-1662. (김경민, 안무혁, 이윤호, "네이버 DB정보 기반 영화 평점 예측 및 악의적 평점 기록자 Filtering", 대한산업공학회 춘계공동학술대회논문집, 2013, 1651-1662.)
36 Li, F., N. Liu, H. Jin, K. Zhao, Q. Yang, and X. Zhu, "Incorporating Reviewer, and Product Information for Review Rating Prediction", 22nd International Joint Conferences on Artificial Intelligence, Vol.3, 2011, 1820-1825.
37 Landauer, T.K., P.W. Foltz, and D. Laham, "An Introduction to Latent Semantic Analysis", Discourse Processes, Vol.25, No.2-3, 1998, 259-284. doi : http://dx.doi.org/10.1080/01638539809545028 (Downloaded August 30, 2017).   DOI
38 Lee, J., D.H. Park, and I. Han, "The Effect of Online Negative Online Consumer Reviews on Product Attitude : An Information Processing View", Electronic Commerce Research, and Applications, Vol.7, No.3, 2008, 341-352.   DOI
39 Lee, S., T.R. Jeon, G.D. Baek, and S.S. Kim, "A Movie Rating Prediction System of User Propensity Analysis based on Collaborative Filtering, and Fuzzy System", Journal of Korean Institute of Intelligent Systems, Vol.19, No.2, 2009, 242-247. (이수진, 전태룡, 백경동, 김성신, "협업적 필터링 및 퍼지시스템 기반 사용자 성향분석에 의한 영화 평가 예측 시스템", 한국지능시스템학회논문지, 제19권, 제2호, 2009, 242-247.)   DOI
40 Liu, C. and N. Kim "Methodology for Improving the Reliability of the Rating System for Leisure Activity Information Sites : Focusing on a Movie Information Site", Journal of Tourism & Leisure Research, Vol.27, No.7, 2015, 187-200. (류 신, 김남규, "여가활동 정보 사이트의 평점 시스템 신뢰성 향상 방안", 관광레저연구, 제27권, 제7호, 2015, 187-200.)
41 Liu, C.L., W.H. Hsaio, C.H. Lee, G.C. Lu, and E. Jou, "Movie Rating and Review Summarization in Mobile Environment", IEEE Transactions on Systems, Man, and Cybernetics-Part C : Applications and Reviews, Vol.42 No.3, 2012, 397-407.   DOI
42 Marovic, M., M. Mihokovic, M. Miksa, S. Pribil, and A. Tus, "Automatic Movie Ratings Prediction Using Machine Learning", Proceedings of the 34th International Convention, 2011, 1640-1645.