Browse > Article
http://dx.doi.org/10.5859/KAIS.2022.31.4.161

The Impacts of Online Game Reviews' Characteristics on Review Helpfulness: Based on Topic Modeling Analysis  

Bae, Sung Hun (경북대학교 경영학부)
Kim, Hyun Mook (경북대학교 경영학부)
Lee, Ui Jun (서울대학교 경영학과)
Lee, Sae Rom (경북대학교 경영학부)
Publication Information
The Journal of Information Systems / v.31, no.4, 2022 , pp. 161-187 More about this Journal
Abstract
Purpose This study analyzed the topic of game review contents and how the characteristics of game reviews affect the reviews helpfulness. In addition, this study explore the content of game reviews according to the game's sales strategy such as early access strategy and releasing without early access. Design/methodology/approach We collected a list of 3,572 action genre games released in 2020. 58,336 online reviews were collected by random sampling 50 reviews in each games, and topic modeling was performed on those reviews. We dynamized the results of topic modeling and analyzed the effect on review helpfulness with multiple regression analysis. Findings The results of analysis indicate that the longer the review is or the shorter the time it is written, the more helpful the review is. In addition the topic with positive and negative review has a significant effect on the review helpfulness. As a result of exploratory analysis, games from early access had relatively fewer reviews of story-related topics than games that were released without early access. These findings can present direct guidelines for collecting specific opinions from customers in the game industry when releasing games.
Keywords
Online Review; Online Game Review; Experience Goods; Review Helpfulness; Topic Modeling; Early-access Strategies;
Citations & Related Records
Times Cited By KSCI : 8  (Citation Analysis)
연도 인용수 순위
1 Vallurupalli, V., and Bose, I., "Exploring Thematic Composition of Online Reviews: A Topic Modeling Approach," Electronic Markets, Vol. 30, No. 4, 2020, pp. 791-804.   DOI
2 Xie, K. L., Chen, C., and Wu, S., "Online Consumer Review Factors Affecting Offline Hotel Popularity: Evidence from Tripadvisor," Journal of Travel & Tourism Marketing, Vol. 33, No. 2, 2016, pp. 211-223.   DOI
3 Yang, Y., Chen, C., and Bao, F. S., "Aspect-Based Helpfulness Prediction for Online Product Reviews," 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI) September, 2016, pp. 836-843.
4 Ye, Q., Law, R., and Gu, B., "The Impact of Online User Reviews on Hotel Room Sales," International Journal of Hospitality Management, Vol. 28, No. 1, 2009, pp. 180-182.
5 Yin, J., and Wang, J., "A Dirichlet Multinomial Mixture Model-Based Approach for Short Text Clustering", In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 233-242.
6 Zhou, S., and Guo, B., "The Order Effect on Online Review Helpfulness: A Social Influence Perspective," Decision Support Systems, Vol. 93, 2017, pp. 77-87.
7 Statista, Number of games released on Steam worldwide from 2004 to 2021, Retrieved February 2022, Available: https://www.statista.com/statistics/552623/number-games-released-steam/ 
8 강성안, 김동연, 류민호, "텍스트 마이닝을 이용한 부동산 서비스 앱 리뷰 분석," 정보시스템연구, 제30권, 제4호, 2021, pp. 227-245.
9 고은주, 성경, 최용락, "디지털 컨텐츠 저작권 보안기술 동향" 인터넷정보학회지, 제4권, 제2호, 2003, pp. 39-46.
10 김정웅, 정혜정, 양해술, "CMMI 인증을 위한 게임 소프트웨어 특수성 고찰," 제8회 SW 품질관리 심포지움 논문집, 2004, pp. 448-452.
11 Gao, B., Hu, N., and Bose, I., "Follow the Herd or Be Myself? An Analysis of Consistency in Behavior of Reviewers and Helpfulness of Their Reviews," Decision Support Systems, Vol. 95, 2017, pp. 1-11.   DOI
12 Ghose, A., and Ipeirotis, P. G., "Estimating the Helpfulness and Economic Impact of Product Reviews: Mining Text and Reviewer Characteristics," IEEE Transactions on Knowledge and Data Engineering, Vol. 23, No. 10, 2010, pp. 1498-1512.   DOI
13 Hansen, S. S., Lee, J. K., and Lee, S. Y., "Consumer-Generated Ads on YouTube: Impacts of Source Credibility and Need for Cognition on Attitudes, Interactive Behaviors, and eWOM," Journal of Electronic Commerce Research, Vol. 15, No. 3, 2014, pp. 254.
14 Hao, Y., Ye, Q., Li, Y., and Cheng, Z., "How Does the Valence of Online Consumer Reviews Matter in Consumer Decision Making? Differences between Search Goods and Experience Goods," 2010 43rd Hawaii International Conference on System Sciences, January 2010, pp. 1-10
15 Hlee, S., Lee, J., Yang, S. B., and Koo, C., "An Empirical Examination of Online Restaurant Reviews (Yelp.com): Moderating Roles of Restaurant Type and Self-Image Disclosure," Information and Communication Technologies in Tourism, 2016, pp. 339-353.
16 Hlee, S., Lee, J., Yang, S. B., and Koo, C., "The Moderating Effect of Restaurant Type on Hedonic Versus Utilitarian Review Evaluations," International Journal of Hospitality Management, 2019, Vol. 77, pp. 195-206.   DOI
17 Hu, Y. H., and Chen, K., "Predicting Hotel Review Helpfulness: The Impact of Review Visibility, and Interaction Between Hotel Stars and Review Ratings," International Journal of Information Management, Vol. 36, No. 6, 2016, pp. 929-944.   DOI
18 Hu, X., and Yang, Y., "What Makes Online Reviews Helpful in Tourism and Hospitality? a Bare-Bones Meta-Analysis," Journal of Hospitality Marketing and Management, Vol. 30, No. 2, 2021, pp. 139-158.
19 Schultes, P., Dorner, V., and Lehner, F., "Leave A Comment! An In-Depth Analysis of User Comments on YouTube," In Proceedings of Wirtscharfsinformatik, Vol. 42, 2013, pp. 659-673.
20 Siering, M., Muntermann, J., and Rajagopalan, B., "Explaining and Predicting Online Review Helpfulness: The Role of Content and Reviewer-Related Signals." Decision Support Systems, Vol. 108, 2018, pp. 1-12.   DOI
21 Smith, E. R., and DeCoster, J., "Dual Process Models in Social and Cognitive Psychology Conceptual Integration and Links to Underlying Memory Systems" Personality and Social Psychology Review, Vol. 4, 2000, pp. 108-131.   DOI
22 민진영, 이애리, "'좋아요'와 '싫어요'같은 간접적 사회적 정보의 방향과 강도는 온라인 뉴스 콘텐츠 댓글의 숙의의 질과 어떤 관련이 있는가? 토픽 모델링을 이용한 토픽 다양성 분석," 정보시스템연구, 제30권, 제4호, 2021, pp. 303-327.
23 김진화, 변현수, 이승훈, "온라인 리뷰를 활용한 사용자 이해 및 서비스 가치 증대," 정보시스템연구, 제20권, 제2호, 2011, pp. 21-36.
24 남영호, "한국 온라인게임 산업의 서비스 혁신 패턴 분석," 한국 데이터베이스 학회지 제15권, 제1호, 2008, pp. 117-137.
25 남현우, "게임 플랫폼과 게임 이용 형태의 변화에 따른 포스트 코로나 시대의 게임산업 경쟁력 방안 연구," 융복합지식학회논문지, 제8권, 제4호, 2020, pp. 137-147.   DOI
26 윤상훈, 김근형, "Word2Vec를 이용한 토픽모델링의 확장 및 분석사례," 정보시스템연구, 제30권, 제1호, 2021, pp. 45-64
27 이의준, 원형식, 이새롬, "숙박공유 플랫폼 서비스의 서비스 공급자 특성이 사용자의 만족감에 미치는 영향: 에어비앤비(Airbnb) 호스트 주체를 중심으로," 정보시스템연구, 제29권, 제1호, 2020, pp. 01-22.
28 장준희, 임형철, 김효남, "얼리액세스 게임의 성공 전략에 관한 연구," 한국컴퓨터정보학회 학술발표논문집, 제28권, 제2호, 2020, pp. 685-688.
29 Ying, W. M., "온라인 리뷰가 모바일 게임 성과에 미치는 영향," 이화여자대학교 석사과정 학위논문, 2021.
30 Ahmad, S. N., and Laroche, M., "How do Expressed Emotions Affect the Helpfulness of a Product Review? Evidence from Reviews Using Latent Semantic Analysis," International Journal of Electronic Commerce, Vol. 20, No. 1, 2015, pp. 76-111.   DOI
31 Annisa, R., and Surjandari, I., "Opinion Mining on Mandalika Hotel Reviews Using Latent Dirichlet Allocation," Procedia Computer Science, Vol. 161, 2019, pp. 739-746.
32 Arafat, M., Qusef, A., and Al-Taher, S, "Steam's Early Access Model: A Study on Consumers' Perspective," In 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), April 2019, pp. 336-341.
33 Kang, H. N., Yong, H. R., and Hwang, H. S., "A Study of Analyzing on Online Game Reviews Using a Data Mining Approach: Steam Community Data," International Journal of Innovation, Management and Technology, Vol. 8, No. 2, 2017, pp. 90.
34 Hwang, Y., Choi, S., and Mattila, A. S., "The Role of Dialecticism and Reviewer Expertise in Consumer Responses to Mixed Reviews," International Journal of Hospitality Management, Vol. 69, 2018, pp. 49-55.   DOI
35 Jensen, M. L., Averbeck, J. M., Zhang, Z., and Wright, K. B., "Credibility of anonymous online product reviews: A language expectancy perspective," Journal of Management Information Systems, Vol. 30, No. 1, 2013, pp. 293-324.
36 Jia, H., Shin, S., and Jiao, J., "Does the Length of a Review Matter in Perceived Helpfulness? The Moderating Role of Product Experience," Journal of Research in Interactive Marketing, Vol. 16, No. 2, 2021, pp. 221-236.
37 Khern-am-nuai, W., Kannan, K., and Ghasemkhani, H., "Extrinsic Versus Intrinsic Rewards for Contributing Reviews in an Online Platform," Information Systems Research, Vol. 29, No. 4, 2018, pp. 871-892.   DOI
38 King, R. A., Racherla, P., and Bush, V. D., "What We Know and Don't Know about Online Word-of-Mouth: A Review and Synthesis of the Literature," Journal of Interactive Marketing, Vol. 28, No. 3, 2014, pp. 167-183.
39 Kirilenko, A. P., Stepchenkova, S. O., and Dai, X., "Automated Topic Modeling of Tourist Reviews: Does the Anna Karenina Principle Apply?," Tourism Management, Vol. 83,
40 Korfiatis, N., Garcia-Bariocanal, E., and Sanchez-Alonso, S., "Evaluating Conntent Quality and Helpfulness of Online Product Reviews: The Interplay of Review Helpfulness vs. Review Content," Electronic Commerce Research and Applications, Vol. 11, No. 3, 2012, pp. 205-217.   DOI
41 Surian, D., Nguyen, D. Q., Kennedy, G., Johnson, M., Coiera, E., and Dunn, A. G., "Characterizing Twitter Discussions about HPV Vaccines Using Topic Modeling and Community Detection," Journal of Medical Internet Research, Vol. 18, No. 8, 2016, e6045.
42 Son, J., Negahban, A., Lee, Y., Connolly, J., and Chiang, D., "When More is More and Less is More: Depth and Breadth of Product Reviews and Their Effects on Review Helpfulness." In Proceedings of the 53rd Hawaii International Conference on System Sciences, January 2020.
43 Song, T., Huang, J., Tan, Y., and Yu, Y., "Using User- and Marketer-Generated Content for Box Office Revenue Prediction: Differences between Microblogging and Third-Party Platforms," Information Systems Research, Vol. 30, No. 1, 2019, pp. 191-203
44 Strack, F., and Deutsch, R., "Reflective and Impulsive Determinants of Social Behavior," Personality and Social Psychology Review, Vol. 8, No. 3, 2004, pp. 220-247.   DOI
45 Sutherland, I., Sim, Y., Lee, S. K., Byun, J., and Kiatkawsin, K., "Topic Modeling of Online Accommodation Reviews Via Latent Dirichlet Allocation," Sustainability, Vol. 12, No. 5, 2020, 1821.
46 신대영, "온라인 게임의 온라인 광고 유형 사례," 한국컴퓨터정보학회지, 제17권 제2호, 2009, pp. 29-37.
47 Archak, N., Ghose, A., and Ipeirotis, P. G., "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Management Science, Vol. 57, No. 8, 2011, pp. 1485-1509.   DOI
48 Baek, H., Ahn, J., and Choi, Y., "Helpfulness of Online Consumer Reviews: Readers' Objectives and Review cues," International Journal of Electronic Commerce, Vol. 17, No. 2, 2012, pp. 99-126.
49 Baek, H., Lee, S., Oh, S., and Ahn, J., "Normative Social Influence and Online Review Helpfulness: Polynomial Modeling and Response Surface Analysis," Journal of Electronic Commerce Research, Vol. 16, No. 4, 2015, pp. 290.
50 야오즈옌, 박영기, 홍태호, "리뷰어의 속성이 온라인 리뷰 유용성에 미치는 영향에 관한 연구," 정보시스템연구, 제29권, 제2호, 2020, pp. 173-195.
51 Banerjee, S., Bhattacharyya, S. and Bose, I., "Whose Online Reviews to Trust? Understanding Reviewer Trustworthiness and Its Impact on Business," Decision Support Systems, Vol.96, 2017, pp.17-26.
52 Bi, N. C., "YouTube Product Review Videos as eWOM," The Audience and Business of YouTube and Online Videos, Louisa, H. (eds.), Lexington Books, Lanham, 2018, pp. 59-71.
53 Bickart, B., and Schindler, R. M., "Internet Forums as Influential Sources of Consumer Information," Journal of Interactive Marketing, Vol. 15, No. 3, 2001, pp. 31-40.   DOI
54 Bounie, D., Bourreau, M., Gensollen, M., and Waelbroeck, P., "Do Online Customer Reviews Matter? Evidence from the Video Game Industry," Telecom ParisTech Working Paper, 2008.
55 Brunt, C. S., King, A. S., and King, J. T., "The Influence of User-Generated Content on Video Game Demand," Journal of Cultural Economics, Vol. 44, No. 1, 2020, pp. 35-56.
56 Lin, D., Bezemer, C. P., and Hassan, A. E., "An Empirical Study of Early Access Games on the Steam Platform," Empirical Software Engineering, Vol. 23, No. 2, 2018, pp. 771-799.
57 Kuan, K. K., Hui, K. L., Prasarnphanich, P., and Lai, H. Y., "What Makes a Review Voted? An Empirical Investigation of Review Voting in Online Review Systems." Journal of the Association for Information Systems, Vol. 16, No. 1, 2015, pp. 1.
58 Kwok, L., and Xie, K. L., "Factors Contributing to the Helpfulness of Online hotel Reviews: Does Manager Response Play a Role?," International Journal of Contemporary Hospitality Management, Vol. 28, No. 10, 2016, pp. 2156-2177   DOI
59 Kwon, H. J., Ban, H. J., Jun, J. K., and Kim, H. S., "Topic Modeling and Sentiment Analysis of Online Review for Airlines," Information, Vol. 12, No. 2, 2021, pp. 78.
60 Liu, Z., and Park, S., "What Makes a Useful Online Review? Implication for Travel Product Websites," Tourism Management, Vol. 47, 2015, pp. 140-151.
61 Lee, I., "Usefulness, Funniness, and Coolness Votes of Viewers: An Analysis of Social Shoppers' Online Reviews," Industrial Management and Data Systems, Vol. 118, No. 4, 2018, pp. 700-713   DOI
62 Luan, J., Yao, Z., Zhao, F., and Liu, H., "Search Product and Experience Product Online Reviews: An Eye-Tracking Study on Consumers' Review Search Behavior," Computers in Human Behavior, Vol. 65, 2016, pp. 420-430.   DOI
63 Malik, M. S. I., and Hussain, A.,"Exploring the Influential Reviewer, Review and Product Determinants for Review Helpfulness," Artificial Intelligence Review, Vol. 53, No. 1, 2020, pp. 407-427.
64 Mudambi, S.M. and Schuff, D., "What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.com", MIS Quarterly, Vol. 34, No. 1 2010, pp. 185-200.   DOI
65 Chu, W. J., and Roh, M. J., "Exploring the Role of Preference Heterogeneity and Causal Attribution in Online Ratings Dynamics," Asia Marketing Journal, Vol. 15, No. 4, 2014, pp. 61-101.
66 Chen, L., Choi, K. J., and Lee, J. Y., "The Effect Of Rating Dispersion on Purchase of Experience Goods Based on the Korean Movie Box Office Data," Asia Marketing Journal, Vol. 21, No. 1, 2019, pp. 1-21.   DOI
67 Cheng, Y. H., and Ho, H. Y., "Social Influence's Impact on Reader Perceptions of Online Reviews," Journal of Business Research, Vol. 68, No. 4, 2015, pp. 883-887.   DOI
68 Chou, Y. C., Chuang, H. H. C., and Liang, T. P., "Elaboration Likelihood Model, Endogenous Quality Indicators, and Online Review Helpfulness," Decision Support Systems, Vol. 153, 2022
69 Chua, A. Y. K., and Banerjee, S., "Understanding Review Helpfulness as A Function of Reviewer Reputation, Review Rating, and Review Depth," Journal of the Association for Information Science and Technology, Vol. 66, No. 2, 2014, pp. 354-362    DOI
70 Clemons, E. K. "How Information Changes Consumer Behavior and How Consumer Behavior Determines Corporate Strategy," Journal of Management Information Systems, Vol. 25, No. 2, 2008, pp. 13-40.
71 Craciun, G., and Moore, K., "Credibility of Negative Online Product Reviews: Reviewer Gender, Reputation and Emotion Effects," Computers in Human Behavior, Vol. 97, 2019, pp. 104-115.
72 Duan, W., Gu, B., and Whinston, A. B., "The Dynamics of Online Word-of-Mouth and Product Sales-An Empirical Investigation of the Movie Industry," Journal of Retailing, Vol. 84, No. 2, 2008, pp. 233-242.   DOI
73 Forman, C., Ghose, A., and Wiesenfeld, B., "Examining the Relationship Between Reviews and Sales: The Role of Reviewer Identity Disclosure in Electronic Markets," Information Systems Research, Vol. 19, No. 3, 2008, pp. 291-313.   DOI
74 Park, S., and Nicolau, J. L., "Asymmetric Effects of Online Consumer Reviews," Annals of Tourism Research, Vol. 50, 2015, pp. 67-83.   DOI
75 Newman, D., Lau, J. H., Grieser, K., and Baldwin, T., "Automatic Evaluation of Topic Coherence," Human language technologies: The 2010 Annual Conference of the North American Chapter of the Association For Computational Linguistics, 2010, pp. 100-108.
76 Nucciarelli, A., Li, F., Fernandes, K. J., Goumagias, N., Cabras, I., Devlin, S., ... and Cowling, P., "From Value Chains to Technological Platforms: The Effects of Crowdfunding in the Digital Game Industry," Journal of Business Research, Vol. 78, 2017, pp. 341-352.
77 Pan, Y., and Zhang, J. Q., "Born Unequal: A Study of the Helpfulness of User-Generated Product Reviews," Journal of Retailing, Vol. 87, No. 4, 2011, pp. 598-612.   DOI
78 Pentina, I., Bailey, A. A., and Zhang, L., "Exploring Effects of Source Similarity, Message Valence, and Receiver Regulatory Focus on Yelp Review Persuasiveness and Purchase Intentions," Journal of Marketing Communications, Vol. 24, No. 2, 2018, pp. 125-145.
79 Perron, B., and Wolf, M. J., (Eds.) "The Video Game Theory Reader 2" Routledge, NewYork, Vol. 2, 2009.
80 Porturas, T., and Taylor, R. A., "Forty Years of Emergency Medicine Research: Uncovering Research Themes and Trends Through Topic Modeling," The American Journal of Emergency Medicine, Vol. 45, 2021, pp. 213-220.
81 Robinson, R., Goh, T. T., and Zhang, R., "Textual Factors in Online Product Reviews: A Foundation For a More Influential Approach to Opinion Mining," Electronic Commerce Research, Vol. 12, No. 3, 2012, pp. 301-330.   DOI
82 Salehan, M., and Kim, D. J., "Predicting the Performance of Online Consumer Reviews: A Sentiment Mining Approach to Big Data Analytics," Decision Support Systems, Vol. 81, 2016, pp. 30-40.