Asia pacific journal of information systems

The Korea Society of Management Information Systems (한국경영정보학회)
권호 표지
DOI QR코드

DOI QR Code

Too Much Information - Trying to Help or Deceive? An Analysis of Yelp Reviews

  • Hyuk Shin (Department of Business Administration, The Catholic University of Korea) ;
  • Hong Joo Lee (Department of Business Administration, The Catholic University of Korea) ;
  • Ruth Angelie Cruz (Decision Sciences and Innovation at De La Salle University (DLSU))
  • Received : 2022.05.30
  • Accepted : 2023.01.11
  • Published : 2023.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

The proliferation of online customer reviews has completely changed how consumers purchase. Consumers now heavily depend on authentic experiences shared by previous customers. However, deceptive reviews that aim to manipulate customer decision-making to promote or defame a product or service pose a risk to businesses and buyers. The studies investigating consumer perception of deceptive reviews found that one of the important cues is based on review content. This study aims to investigate the impact of the information amount of review on the review truthfulness. This study adopted the Information Manipulation Theory (IMT) as an overarching theory, which asserts that the violations of one or more of the Gricean maxim are deceptive behaviors. It is regarded as a quantity violation if the required information amount is not delivered or more information is delivered; that is an attempt at deception. A topic modeling algorithm is implemented to reveal the distribution of each topic embedded in a text. This study measures information amount as topic diversity based on the results of topic modeling, and topic diversity shows how heterogeneous a text review is. Two datasets of restaurant reviews on Yelp.com, which have Filtered (deceptive) and Unfiltered (genuine) reviews, were used to test the hypotheses. Reviews that contain more diverse topics tend to be truthful. However, excessive topic diversity produces an inverted U-shaped relationship with truthfulness. Moreover, we find an interaction effect between topic diversity and reviews' ratings. This result suggests that the impact of topic diversity is strengthened when deceptive reviews have lower ratings. This study contributes to the existing literature on IMT by building the connection between topic diversity in a review and its truthfulness. In addition, the empirical results show that topic diversity is a reliable measure for gauging information amount of reviews.

Keywords

Acknowledgement

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2020S1A3A2A02093277).

References

  1. Ansari, S., and Gupta, S. (2021). Review manipulation: literature review, and future research agenda. Pacific Asia Journal of the Association for Information Systems, 13(1), 97-121. https://doi.org/10.17705/1pais.13104
  2. Aprilia, M. P. (2018). Instagram and buying decision processes of restaurant consumers. AJMC (Asian Journal of Media and Communication), 2(2), 13-18.
  3. Azarbonyad, H., Dehghani, M., Kenter, T., Marx, M., Kamps, J., and de Rijke, M. (2017). Hierarchical re-estimation of topic models for measuring topical diversity. In Advances in Information Retrieval (pp. 68-81). Cham: Springer International Publishing.
  4. Banerjee, S., and Chua, A. Y. K. (2014). A study of manipulative and authentic negative reviews. In Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication (pp. 1-6). New York, NY, USA: Association for Computing Machinery.
  5. Barbado, R., Araque, O., and Iglesias, C. A. (2019). A framework for fake review detection in online consumer electronics retailers. Information Processing & Management, 56(4), 1234-1244. https://doi.org/10.1016/j.ipm.2019.03.002
  6. Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). Latent dirichlet allocation. The Journal of Machine Learning Research, 3, 993-1022.
  7. Boush, D. M., Friestad, M., and Wright, P. (2015). Deception in the marketplace: The psychology of deceptive persuasion and consumer self-protection, Routledge.
  8. Buller, D. B., and Burgoon, J. K. (1996). Interpersonal Deception Theory. Communication Theory, 6(3), 203-242. https://doi.org/10.1111/j.1468-2885.1996.tb00127.x
  9. Buller, D. B., Burgoon, J. K., Buslig, A., and Roiger, J. (1996). Testing interpersonal deception theory: The language of interpersonal deception. Communication Theory, 6(3), 268-288. https://doi.org/10.1111/j.1468-2885.1996.tb00129.x
  10. Burgoon, J. K., and Qin, T. (2006). The dynamic nature of deceptive verbal communication. Journal of Language and Social Psychology, 25(1), 76-96. https://doi.org/10.1177/0261927X05284482
  11. Chen, Dhanasobhon, S., and Smith, M. D. (2008). All reviews are not created equal: The disaggregate impact of reviews and reviewers at Amazon.com (SSRN Scholarly Paper No. ID 918083). Rochester, NY: Social Science Research Network.
  12. Chernyaeva, O., Kim, E., and Hong, T. (2021). The Detection of well-known and unknown Brands' products with manipulated reviews using sentiment analysis. Asia Pacific Journal of Information Systems, 31(4), 472-490. https://doi.org/10.14329/apjis.2021.31.4.472
  13. Chevalier, J. A., and Mayzlin, D. (2006). The effect of word of mouth on sales: online book reviews. Journal of Marketing Research, 43(3), 345-354. https://doi.org/10.1509/jmkr.43.3.345
  14. Cho, M., Hwang, D., and Jeon, S. (2022). The impact of topic distribution on review sentiment: A comparative study between South Korea and the U.S.. Asia Pacific Journal of Information Systems, 32(3), 514-536. https://doi.org/10.14329/apjis.2022.32.3.514
  15. Choi, A. A., Cho, D., Yim, D., Moon, J. Y., and Oh, W. (2019). When seeing helps believing: The interactive effects of previews and reviews on e-book purchases. Information Systems Research, 30(4), 1164-1183. https://doi.org/10.1287/isre.2019.0857
  16. Clemons, E. K., Gao, G. G., and Hitt, L. M. (2006). When online reviews meet Hyperdifferentiation: A study of the craft beer industry. Journal of Management Information Systems, 23(2), 149-171. https://doi.org/10.1109/HICSS.2006.534
  17. Comegys, C., Hannula, M., and Vaisanen, J. (2006). Longitudinal comparison of finnish and US online shopping behaviour among university students: The five-stage buying decision process. Journal of Targeting, Measurement and Analysis for Marketing, 14(4), 336-356. https://doi.org/10.1057/palgrave.jt.5740193
  18. Dabholkar, P. A. (2006). Factors influencing consumer choice of a 'rating web site': An experimental investigation of an online interactive decision aid. Journal of Marketing Theory and Practice, 14(4), 259-273. https://doi.org/10.2753/MTP1069-6679140401
  19. Debortoli, S., Muller, O., Junglas, I., and Brocke, J. vom. (2016). Text mining for information systems researchers: An annotated topic modeling tutorial. Communications of the Association for Information Systems, 39(7), 110-135. https://doi.org/10.17705/1CAIS.03907
  20. DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., and Cooper, H. (2003). Cues to deception. Psychological Bulletin, 129(1), 74-118. https://doi.org/10.1037/0033-2909.129.1.74
  21. Dong, L., Ji, S., Zhang, C., Zhang, Q., Chiu, DicksonK. W., Qiu, L., and Li, D. (2018). An unsupervised topic-sentiment joint probabilistic model for detecting deceptive reviews. Expert Systems with Applications, 114, 210-223. https://doi.org/10.1016/j.eswa.2018.07.005
  22. Ekman, P. (2011). Ekman, P. (2009) Telling lies : clues to deceit in the marketplace, politics, and marriage. New York: Norton. American Journal of Clinical Hypnosis, 53(4), 287-288. https://doi.org/10.1080/00029157.2011.10404358
  23. Filieri, R. (2016). What makes an online consumer review trustworthy?. Annals of Tourism Research, 58, 46-64. https://doi.org/10.1016/j.annals.2015.12.019
  24. Filieri, R., Raguseo, E., and Vitari, C. (2018). When are extreme ratings more helpful? Empirical evidence on the moderating effects of review characteristics and product type. Computers in Human Behavior, 88, 134-142. https://doi.org/10.1016/j.chb.2018.05.042
  25. Forman, C., Ghose, A., and Wiesenfeld, B. (2008). Examining the relationship between reviews and sales: The role of reviewer identity disclosure in electronic markets. Information Systems Research, 19(3), 291-313. https://doi.org/10.1287/isre.1080.0193
  26. Fuller, C. M., Biros, D. P., Burgoon, J., and Nunamaker, J. (2013). An examination and validation of linguistic constructs for studying high-stakes deception. Group Decision and Negotiation, 22(1), 117-134. https://doi.org/10.1007/s10726-012-9300-z
  27. Ghasemaghaei, M., Eslami, S. P., Deal, K., and Hassanein, K. (2018). Reviews' length and sentiment as correlates of online reviews' ratings. Internet Research, 28(3), 544-563. https://doi.org/10.1108/IntR-12-2016-0394
  28. Grice, P. (1989). Studies in the way of words. Harvard University Press.
  29. Harris. (1954). Distributional structure. Word, 10(2-3), 146-162. https://doi.org/10.1080/00437956.1954.11659520
  30. Harris, C. G. (2012). Detecting deceptive opinion spam using human computation. In AAAI Conference on Artificial Intelligence. Vol. Technical Report WS-12-08, AAAI, pp. 87-93.
  31. Hu, N., Bose, I., Koh, N. S., and Liu, L. (2012). Manipulation of online reviews: An analysis of ratings, readability, and sentiments. Decision Support Systems, 52(3), 674-684. https://doi.org/10.1016/j.dss.2011.11.002
  32. Huang, A. H., Chen, K., Yen, D. C., and Tran, T. P. (2015). A study of factors that contribute to online review helpfulness. Computers in Human Behavior, 48: 17-27. https://doi.org/10.1016/j.chb.2015.01.010
  33. Hutto, C., and Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the International AAAI Conference on Web and Social Media, 8(1), 216-225.
  34. Jelodar, H., Wang, Y., Yuan, C., Feng, X., Jiang, X., Li, Y., and Zhao, L. (2019). Latent Dirichlet Allocation (LDA) and topic modeling: models, applications, a surve. Multimedia Tools and Applications, 78(11), 15169-15211. https://doi.org/10.1007/s11042-018-6894-4
  35. Jiang, Z., and Benbasat, I. (2004). Virtual product experience: Effects of visual and functional control of products on perceived diagnosticity and flow in electronic shopping. Journal of Management Information Systems, 21(3), 111-147.
  36. Jindal, N., and Liu, B. (2008). Opinion spam and analysis. In 2008 International Conference on Web Search and Data Mining, pp. 219-230.
  37. Kohli, R., Devaraj, S., and Mahmood, M. A. (2014). Understanding determinants of online consumer satisfaction: A decision process perspective. Journal of Management Information Systems, 21(1), 115-136. https://doi.org/10.1080/07421222.2004.11045796
  38. Kotler, P. (2000). Marketing Management Millenium Edition, Tenth Edition, Vol. Tenth Edition, Prentice-Hall.
  39. Kumar, N., Venugopal, D., Qiu, L., and Kumar, S. (2018). Detecting review manipulation on online platforms with hierarchical supervised learning. Journal of Management Information Systems, 35(1), 350-380. https://doi.org/10.1080/07421222.2018.1440758
  40. Kwon, H.-J., Ban, H.-J., Jun, J.-K., and Kim, H.-S. (2021). Topic modeling and sentiment analysis of online review for airlines. Information, 12(2), 78.
  41. Lappas, T. (2012). Fake reviews: The malicious perspective. In Natural Language Processing and Information Systems. Presented at the International Conference on Application of Natural Language to Information Systems. Springer, Berlin, Heidelberg, pp. 23-34.
  42. Lee, J., and Lee, H. J. (2016). Your expectation matters when you read online consumer reviews: The review extremity and the escalated confirmation effect. Asia Pacific Journal of Information Systems, 26(3), 449-476. https://doi.org/10.5859/KAIS.2016.25.2.173
  43. Li, L., Fu, L., and Zhang, W. (2022). Impact of text diversity on review helpfulness: A topic modeling approach. Interdisciplinary Journal of Information, Knowledge, and Management, 17, 87-100. https://doi.org/10.28945/4922
  44. Lim, J., and Lee, H. C. (2020). Comparisons of service quality perceptions between full service carriers and low cost carriers in airline travel. Current Issues in Tourism, 23(10), 1261-1276. https://doi.org/10.1080/13683500.2019.1604638
  45. Liu, Lee, D., and Srinivasan, K. (2019). Large-scale cross-category analysis of consumer review content on sales conversion leveraging deep learning. Journal of Marketing Research, 56(6), 918-943. https://doi.org/10.1177/0022243719866690
  46. Liu and Park, S. (2015). What makes a useful online review? Implication for travel product websites. Tourism Management, 47, 140-151. https://doi.org/10.1016/j.tourman.2014.09.020
  47. Luca, M. (2016). Reviews, reputation, and revenue: the case of Yelp.com (SSRN Scholarly Paper No. ID 1928601). Rochester, NY: Social Science Research Network, p. 41 pages.
  48. Martinez-Torres, M. R., and Toral, S. L. (2019). A machine learning approach for the identification of the deceptive reviews in the hospitality sector using unique attributes and sentiment orientation, Tourism Management 75: 393-403. https://doi.org/10.1016/j.tourman.2019.06.003
  49. McCornack. (1992). Information manipulation theory. Communication Monographs, 59(1), 1-16. https://doi.org/10.1080/03637759209376245
  50. Metts, S. (2016). An exploratory investigation of deception in close relationships. Journal of Social and Personal Relationships, 6(2), 159-197. https://doi.org/10.1177/026540758900600202
  51. Mudambi, S., and Schuff, D. (2010). What makes a helpful online review? A study of customer reviews on Amazon.com. MIS Quarterly, 34(1), 185-200. https://doi.org/10.2307/20721420
  52. Ong, T., Mannino, M., and Gregg, D. (2014). Linguistic characteristics of shill reviews. Electronic Commerce Research and Applications, 13(2), 69-78. https://doi.org/10.1016/j.elerap.2013.10.002
  53. Ott, M., Cardie, C., and Hancock, J. (2012). Estimating the prevalence of deception in online review communities. In Proceedings of the 21st International Conference on World Wide Web. New York, NY, USA: Association for Computing Machinery, pp. 201-210.
  54. Pan, Y., and Zhang, J. Q. (2011). Born unequal: A study of the helpfulness of user-generated product reviews. Journal of Retailing, 87(4), 598-612. https://doi.org/10.1016/j.jretai.2011.05.002
  55. Peng, L., Cui, G., Zhuang, M., and Li, C. (2016). Consumer perceptions of online review deceptions: an empirical study in China. Journal of Consumer Marketing, 33(4), 269-280. https://doi.org/10.1108/JCM-01-2015-1281
  56. Pennebaker, J. W., Francis, M. E., and Booth, R. J. (2001). Linguistic inquiry and word count: LIWC 2001, Mahway: Lawrence Erlbaum Associates, 71(2001), 2001.
  57. Rayana, S., and Akoglu, L. (2016). Collective opinion spam detection using active inference. In Proceedings of the 2016 SIAM International Conference on Data Mining (SDM). Society for Industrial and Applied Mathematics, pp. 630-638.
  58. Rucker, D. D., Tormala, Z. L., Petty, R. E., and Brinol, P. (2014). Consumer conviction and commitment: An appraisal based framework for attitude certainty. Journal of Consumer Psycholog, 24(1), 119-136. https://doi.org/10.1016/j.jcps.2013.07.001
  59. Sarkadi, S. (2018). Deception, International Joint Conference on Artificial Intelligence. 5781-5782.
  60. Shan, G., Zhou, L., and Zhang, D. (2021). From conflicts and confusion to doubts: Examining review inconsistency for fake review detection. Decision Support Systems, 144, 1-11. https://doi.org/10.1016/j.dss.2021.113513
  61. Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379-423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
  62. Shin, S., Du, Q., Ma, Y., Fan, W., and Xiang, Z. (2021). Moderating effects of rating on text and helpfulness in online hotel reviews: an analytical approach. Journal of Hospitality Marketing & Management, 30(2), 159-177. https://doi.org/10.1080/19368623.2020.1778596
  63. Si, L., and Callan, J. (2001). A statistical model for scientific readability. In Proceedings of the Tenth International Conference on Information and Knowledge Management. New York, NY, USA: Association for Computing Machinery, pp. 574-576.
  64. Son, J., Negahban, A., and Chiang, D. (2019). Topic diversity of online consumer reviews and its effect on review helpfulness. AMCIS 2019 Proceedings. 6.
  65. Sutherland, I., Sim, Y., Lee, S. K., Byun, J., and Kiatkawsin, K. (2020). Topic modeling of online accommodation reviews via latent dirichlet allocation. Sustainability, 12(5), 1821-1836. https://doi.org/10.3390/su12051821
  66. Turner, R. E., Edgley, C., and Olmstead, G. (1975). Information control in conversations: Honesty is not always the best policy. The Kansas Journal of Sociology, 11(1), 69-89. https://doi.org/10.17161/STR.1808.6098
  67. Turney, P. D., and Pantel, P. (2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37, 141-188. https://doi.org/10.1613/jair.2934
  68. Vidanagama, D. U., Silva, T. P., and Karunananda, A. S. (2020). Deceptive consumer review detection: a survey. Artificial Intelligence Review, 53(2), 1323-1352. https://doi.org/10.1007/s10462-019-09697-5
  69. Wang, F., and Karimi, S. (2019). This product works well (for me), The impact of first-person singular pronouns on online review helpfulness. Journal of Business Research, 104, 283-294. https://doi.org/10.1016/j.jbusres.2019.07.028
  70. Wang, Miller, M. J., Schmitt, M. R., and Wen, F. K. (2013). Assessing readability formula differences with written health information materials: Application, results, and recommendations. Research in Social and Administrative Pharmacy, 9(5), 503-516. https://doi.org/10.1016/j.sapharm.2012.05.009
  71. Wu, C., Che, H., Chan, T. Y., and Lu, X. (2015). The economic value of online reviews. Marketing Science, 34(5), 739-754. https://doi.org/10.1287/mksc.2015.0926
  72. Yoo, K.-H., and Gretzel, U. (2009). Comparison of Deceptive and Truthful Travel Reviews, pp. 37-47.
  73. Zhang, W., Du, Y., Yoshida, T., and Wang, Q. (2018). DRI-RCNN: An approach to deceptive review identification using recurrent convolutional neural network. Information Processing & Management, 54(4), 576-592. https://doi.org/10.1016/j.ipm.2018.03.007
  74. Zhong, Q., Liang, S., Cui, L., Chan, H. K., and Qiu, Y. (2018). Using online reviews to explore consumer purchasing behaviour in different cultural settings. Kybernetes, 48(6), 1242-1263. https://doi.org/10.1108/K-03-2018-0117
  75. Zhou, L., Twitchell, D. P., Qin, T., Burgoon, J. K., and Nunamaker, J. F. (2003). An exploratory study into deception detection in text-based computer-mediated communication, In 36th Annual Hawaii International Conference on System Sciences, 2003. p. 10.
  76. Zhou, Lina, Burgoon, J. K., Nunamaker, J. F., and Twitchell, D. (2004). Automating linguistics-based cues for detecting deception in text-based asynchronous computer-mediated communications. Group Decision and Negotiation, 13(1), 81-106. https://doi.org/10.1023/B:GRUP.0000011944.62889.6f