Journal of Intelligence and Information Systems (지능정보연구)

Korea Intelligent Information System Society (한국지능정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

The Effects of Sentiment and Readability on Useful Votes for Customer Reviews with Count Type Review Usefulness Index

온라인 리뷰의 감성과 독해 용이성이 리뷰 유용성에 미치는 영향: 가산형 리뷰 유용성 정보 활용

  • Received : 2015.11.25
  • Accepted : 2016.02.26
  • Published : 2016.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Customer reviews help potential customers make purchasing decisions. However, the prevalence of reviews on websites push the customer to sift through them and change the focus from a mere search to identifying which of the available reviews are valuable and useful for the purchasing decision at hand. To identify useful reviews, websites have developed different mechanisms to give customers options when evaluating existing reviews. Websites allow users to rate the usefulness of a customer review as helpful or not. Amazon.com uses a ratio-type helpfulness, while Yelp.com uses a count-type usefulness index. This usefulness index provides helpful reviews to future potential purchasers. This study investigated the effects of sentiment and readability on useful votes for customer reviews. Similar studies on the relationship between sentiment and readability have focused on the ratio-type usefulness index utilized by websites such as Amazon.com. In this study, Yelp.com's count-type usefulness index for restaurant reviews was used to investigate the relationship between sentiment/readability and usefulness votes. Yelp.com's online customer reviews for stores in the beverage and food categories were used for the analysis. In total, 170,294 reviews containing information on a store's reputation and popularity were used. The control variables were the review length, store reputation, and popularity; the independent variables were the sentiment and readability, while the dependent variable was the number of helpful votes. The review rating is the moderating variable for the review sentiment and readability. The length is the number of characters in a review. The popularity is the number of reviews for a store, and the reputation is the general average rating of all reviews for a store. The readability of a review was calculated with the Coleman-Liau index. The sentiment is a positivity score for the review as calculated by SentiWordNet. The review rating is a preference score selected from 1 to 5 (stars) by the review author. The dependent variable (i.e., usefulness votes) used in this study is a count variable. Therefore, the Poisson regression model, which is commonly used to account for the discrete and nonnegative nature of count data, was applied in the analyses. The increase in helpful votes was assumed to follow a Poisson distribution. Because the Poisson model assumes an equal mean and variance and the data were over-dispersed, a negative binomial distribution model that allows for over-dispersion of the count variable was used for the estimation. Zero-inflated negative binomial regression was used to model count variables with excessive zeros and over-dispersed count outcome variables. With this model, the excess zeros were assumed to be generated through a separate process from the count values and therefore should be modeled as independently as possible. The results showed that positive sentiment had a negative effect on gaining useful votes for positive reviews but no significant effect on negative reviews. Poor readability had a negative effect on gaining useful votes and was not moderated by the review star ratings. These findings yield considerable managerial implications. The results are helpful for online websites when analyzing their review guidelines and identifying useful reviews for their business. Based on this study, positive reviews are not necessarily helpful; therefore, restaurants should consider which type of positive review is helpful for their business. Second, this study is beneficial for businesses and website designers in creating review mechanisms to know which type of reviews to highlight on their websites and which type of reviews can be beneficial to the business. Moreover, this study highlights the review systems employed by websites to allow their customers to post rating reviews.

온라인 쇼핑몰의 상품에 대한 고객 리뷰는 구매자들의 구매 의사결정에 영향을 미치고 있으며 중요한 구전효과의 원천과 의사결정의 정보 원천의 역할을 하고 있다. 한 제품에 대한 리뷰가 무척 많기에 온라인 쇼핑몰들은 고객 리뷰 평가 방안을 도입하였고, 이를 통해 고객들에게 유용하리라고 판단되는 리뷰들을 걸러서 보여주거나 강조할 수 있게 되었다. 리뷰 평가 방안은 해당 리뷰가 도움이 되었는지 혹은 도움이 되지 않았는 지를 리뷰를 읽은 고객이 평가하게 하는 방안이다. Amazon.com은 고객 평가를 바탕으로 총 투표 수 중에서 유용하다는 투표 수의 비율을 리뷰 유용성 지표로 삼고 있으며, Yelp.com은 유용하다는 투표 수 자체를 유용성 지표로 삼고 있다. 본 연구는 고객 리뷰의 감성과 독해 용이성이 리뷰의 유용성에 미치는 영향을 파악하고자 한다. Amazon.com의 고객 리뷰 자료를 활용하여 비율형 유용성 지표를 종속변수로 하는 유사한 연구들이 수행되어 왔다. 본 연구에서는 Yelp.com의 리뷰 자료를 활용하여 가산형 리뷰 유용성 지표인 경우에도 동일한 효과가 존재하는지를 검토하고자 한다. Yelp.com의 음료와 음식 카테고리에 해당하는 업종에 대한 리뷰를 자료로 활용하였으며, 점포의 명성과 인기도 데이터를 파악할 수 있는 170,294개의 리뷰를 분석에 활용하였다. 분석결과는 리뷰의 긍정 정도는 유용 투표수를 늘리는데 음의 영향을 미쳤다. 평가가 긍정적인 리뷰에서는 음의 영향관계가 유의 하였으나, 평가가 부정적인 리뷰에서는 리뷰의 긍정 정도가 유용 투표 수에 미치는 영향은 유의하지 않았다. 독해 용이성은 리뷰가 읽기 어려울 수록 높은 값을 갖으며, 독해의 어려운 정도는 유용 투표수 획득에 음의 영향을 미쳤다. 독해 용이성은 긍정 리뷰, 부정 리뷰 관계없이 모두 음의 영향을 미치는 것으로 분석되었다. 이 결과는 유용 투표수가 0인 리뷰를 포함하여 영과잉 음이항 회귀분석을 수행한 경우와 유용 투표수가 0인 리뷰를 제외하고 음이항 회귀분석을 수행한 경우 모두 동일하게 파악되었다.

Keywords

References

  1. Baccianella, S., A. Esuli, and F. Sehastiani, "SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining," Proceedings of the Seventh Conference on International Language Resources and Evaluation, (2010), 2200-2204.
  2. Baek, H., J. Ahn, and Y. Choi, "Helpfulness of online consumer reviews: Readers' objectives and review cues," International Journal of Electronic Commerce, Vol.17, No.2(2012-13), 99-126.
  3. Baum, S. and M. Spann, "The Interplay Between Online Consumer Reviews and Recommender Systems: An Experimental Analysis," International Journal of Electronic Commerce, Vol.19, No.1(2014), 129-162. https://doi.org/10.2753/JEC1086-4415190104
  4. Bickart, B. and R. M. Schindler, "Internet forums as influential sources of consumer information," Journal of Interactive Marketing, Vol.15, No.3(2001), 31-40. https://doi.org/10.1002/dir.1014
  5. Chae, S. H., J. Im, and J. Y. Kang, "A Comparative Analysis of Social Commerce and Open Market Using User Reviews in Korean Mobile Commerce," Journal of Intelligence and Information Systems, Vol. 21, No.4(2015), 53-77. https://doi.org/10.13088/JIIS.2015.21.4.053
  6. Chen, P., S. Dhanasobhon, and M. Smith, All Reviews Are Not Created Equal: The Disaggregate Impact of Reviews on Sales on Amaon.com, Working paper, Carnegie Mellon University, 2008, Available at SSRN: http://ssrn.com/abstract=918083 (Downloaded 1 November, 2015).
  7. Chevalier, J. and D. Mayzlin, "The Effect of Word of Mouth on Sales: Online Book Reviews," Journal of Marketing Research, Vol.43, No.3(2006), 345-354. https://doi.org/10.1509/jmkr.43.3.345
  8. Choi, J. W. and H. J. Lee, "The Effects of Customer Product Review on Social Presence in Personalized Recommender Systems," Journal of Intelligence and Information Systems, Vol.17, No.3(2011), 115-130.
  9. Chun, B. G. and H. C. Ahn, "A Collaborative Filtering System Combined with Users' Review Mining : Application to the Recommendation of Smartphone Apps," Journal of Intelligence and Information Systems, Vol.21, No.2(2015), 1-18. https://doi.org/10.13088/jiis.2015.21.2.01
  10. Clemons, E., G. Gao, and L. Hitt, "When Online Reviews Meet Hyperdifferentiation: A Study of the Craft Beer Industry," Journal of Management Information Systems, Vol.23, No.2(2006), 149-171. https://doi.org/10.2753/MIS0742-1222230207
  11. Crowley, A. E. and W. D. Hoyer, "An Integrative Framework for Understanding Two-Sided Persuasion," Journal of Consumer Research, Vol.20, No.4(1994), 561-574. https://doi.org/10.1086/209370
  12. Dabholkar, P., "Factors Influencing Consumer Choice of a 'Rating Web Site': An Experimental Investigation of an Online Interactive Decision Aid," Journal of Marketing Theory and Practice Vol.14, No.4(2006), 259-273. https://doi.org/10.2753/MTP1069-6679140401
  13. Forman, C., A. Ghose, and B. Wiesenfeld, "Examining the Relationship Between Reviews and Sales: The Role of Reviewer Identity Disclosure in Electronic Markets," Information Systems Research, Vol.19, No.3(2008), 291-313. https://doi.org/10.1287/isre.1080.0193
  14. Ghose, A. and P. G. Ipeirotis, "Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics," IEEE Transactions on Knowledge and Data Engineering, Vol.23, No.10(2011), 1498-1512. https://doi.org/10.1109/TKDE.2010.188
  15. Ghose, A. and P. G. Ipeirotis, "Designing Ranking Systems for Consumer Reviews: The Impact of Review Subjectivity on Product Sales and Review Quality," Proceedings of the 16th Annual Workshop on Information Technology and Systems, (2006).
  16. Harris, R. B. and D. Paradice, "An investigation of the computer-mediated communication of emotions," Journal of Applied Sciences Research, Vol.3, No.12(2007), 2081-2090.
  17. Hausman, J., B. H. Hall, and Z. Griliches, "Econometric Models for Count Data with an Application to the Patents-R&D Relationship," Econometrica, Vol.52, No.4(1984), 909-937. https://doi.org/10.2307/1911191
  18. Jiang, Z. and I. Benbasat, "Investigating the influence of the functional mechanisms of online product presentations," Information Systems Research, Vol.18, No.4(2007), 454-470. https://doi.org/10.1287/isre.1070.0124
  19. Jiang, B. J. and K. Srinivasan, "Pricing and Persuasive Advertising in a Differentiated Market," Marketing Letters, forthcoming (2015), 1-10.
  20. Korfiatis, N., E. Garcia-Bariocanal, and S. Sanchez-Alonso, "Evaluating content quality and helpfulness of online product reviews: the interplay of review helpfulness vs. review content," Electronic Commerce Research and Applications, Vol.11, No.3(2012), 205-217. https://doi.org/10.1016/j.elerap.2011.10.003
  21. Krosnick, J. A., D. S. Boninger, Y. C. Chuang, M. K. Berent, and C. G. Camot, "Attitude Strength: One Construct or Many Related Constructs?," Journal of Personality and Social Psychology, Vol.65, No.6(1993), 1132-1151. https://doi.org/10.1037/0022-3514.65.1.32
  22. Kumar, N. and I. Benbasat, "The Influence of Recommendations and Consumer Reviews on Evaluations of Websites," Information Systems Research, Vol.17, No.4(2006), 425-439. https://doi.org/10.1287/isre.1060.0107
  23. Michalke, M., "koRpus - ein R-paket zur textanalyse," Tagung experimentell arbeitender Psychologen(TeaP), (2012).
  24. Mudambi, S. M. and D. Schuff, "What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.com," MIS Quarterly, Vol.34, No.1(2010), 185-200. https://doi.org/10.2307/20721420
  25. Park, D. H. and S. Kim, "The effects of consumer knowledge on message processing of electronic word-of-mouth via online consumer reviews," Electronic Commerce Research and Applications, Vol.7, No.4(2009), 399-410.
  26. Park, D. H., J. Lee, and I. Han, "The effect of on-line consumer reviews on consumer purchasing intention: The moderating role of involvement," International Journal of Electronic Commerce, Vol.11, No.4(2007), 125-148. https://doi.org/10.2753/JEC1086-4415110405
  27. Pavlou, P. and A. Dimoka, "The Nature and Role of Feedback Text Comments in Online Marketplaces: Implications for Trust Building, Price Premiums, and Seller Differentiation," Information Systems Research, Vol.17, No.4(2006), 392-414. https://doi.org/10.1287/isre.1060.0106
  28. Schlosser, A., "Source Perceptions and the Persuasiveness of Internet Word-of-Mouth Communication," in Advances in Consumer Research (32), G. Menon and A. Rao (eds.), Duluth, MN: Association for Consumer Research, 2005, 202-203.
  29. Shen, W., J. H. Yu, and J. Rees, "Competing for Attention: An Empirical Study of Online Reviewers' Strategic Behaviors," MIS Quarterly, Vol.39, No.3(2015), 683-696. https://doi.org/10.25300/MISQ/2015/39.3.08
  30. Resnick, P., R. Zeckhauser, E. Friedman, and K. Kuwabara, "Reputation Systems," Communications of the ACM, Vol.43, No.12(2000), 45-48.
  31. Riordan, M. A. and R. J. Kreuz, "Emotion encoding and interpretation in computer-mediated communication: Reasons for use," Computers in Human Behavior, Vol.26, No.6(2010), 1667-1673. https://doi.org/10.1016/j.chb.2010.06.015
  32. Schindler, R. M. and B. Bickart, "Published word-of-mouth: Referable, consumer-generated information on the Internet," In C.P. Haugtvedt, K.A. Machleit, and R.F. Yalch (eds.), Online Consumer Psychology: Understanding and Influencing Behavior in the Virtual World, Hillsdale, NJ: Lawrence Erlbaum, (2005), 35-61.
  33. Schindler, R. and B. Bickart, "Perceived helpfulness of online consumer reviews: The role of message content and style," Journal of Consumer Behaviour, Vol.11, No.3(2012), 234-243. https://doi.org/10.1002/cb.1372
  34. Treisman, A., "Selective attention in man," British Medical Bulletin, Vol.20, No.1(1964), 12-16. https://doi.org/10.1093/oxfordjournals.bmb.a070274

Cited by

  1. Increasing Accuracy of Classifying Useful Reviews by Removing Neutral Terms vol.22, pp.3, 2016, https://doi.org/10.13088/jiis.2016.22.3.129
  2. Making Sales Strategies Based on the Existing Shopping Reviews vol.1865, pp.4, 2021, https://doi.org/10.1088/1742-6596/1865/4/042047