Asia pacific journal of information systems

The Korea Society of Management Information Systems (한국경영정보학회)
권호 표지
DOI QR코드

DOI QR Code

Investigating the Impact of Discrete Emotions Using Transfer Learning Models for Emotion Analysis: A Case Study of TripAdvisor Reviews

  • Dahee Lee (School of Business, Hanyang University) ;
  • Jong Woo Kim (School of Business, Hanyang University)
  • Received : 2023.10.25
  • Accepted : 2024.01.18
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Online reviews play a significant role in consumer purchase decisions on e-commerce platforms. To address information overload in the context of online reviews, factors that drive review helpfulness have received considerable attention from scholars and practitioners. The purpose of this study is to explore the differential effects of discrete emotions (anger, disgust, fear, joy, sadness, and surprise) on perceived review helpfulness, drawing on cognitive appraisal theory of emotion and expectation-confirmation theory. Emotions embedded in 56,157 hotel reviews collected from TripAdvisor.com were extracted based on a transfer learning model to measure emotion variables as an alternative to dictionary-based methods adopted in previous research. We found that anger and fear have positive impacts on review helpfulness, while disgust and joy exert negative impacts. Moreover, hotel star-classification significantly moderates the relationships between several emotions (disgust, fear, and joy) and perceived review helpfulness. Our results extend the understanding of review assessment and have managerial implications for hotel managers and e-commerce vendors.

Keywords

Acknowledgement

This research was conducted in 2020 with the support of the Ministry of Education and the Korea Research Foundation (NRF-2020S1A3A2A02093277).

References

  1. Ahmad, S. N., and Laroche, M. (2015). How do expressed emotions affect the helpfulness of a product review? Evidence from reviews using latent semantic analysis. International Journal of Electronic Commerce, 20(1), 76-111. https://doi.org/10.1080/10864415.2016.1061471 
  2. Baek, H., Ahn, J., and Choi, Y. (2012). Helpfulness of online consumer reviews: Readers' objectives and review cues. International Journal of Electronic Commerce, 17(2), 99-126. https://doi.org/10.2753/JEC1086-4415170204 
  3. Banerjee, S., and Chua, A. Y. (2019). Trust in online hotel reviews across review polarity and hotel category. Computers in Human Behavior, 90, 265-275. https://doi.org/10.1016/j.chb.2018.09.010 
  4. Banerjee, S., Chua, A. Y., and Kim, J. J. (2017). Don't be deceived: Using linguistic analysis to learn how to discern online review authenticity. Journal of the Association for Information Science and Technology, 68(6), 1525-1538. https://doi.org/10.1002/asi.23784 
  5. Berger, J., and Milkman, K. L. (2012). What makes online content viral? Journal of Marketing Research, 49(2), 192-205. https://doi.org/10.1509/jmr.10.0353 
  6. Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with The Natural Language Toolkit. O'Reilly Media, Inc. 
  7. BrightLocal. (2022). Local consumer review survey 2022, Retrieved from https://www.brightlocal.com/research/local-consumer-review-survey/ 
  8. Callan, R. J. (1998). Attributional analysis of customers' hotel selection criteria by UK grading scheme categories. Journal of Travel Research, 36(3), 20-34. https://doi.org/10.1177/004728759803600303 
  9. Cao, Q., Duan, W., and Gan, Q. (2011). Exploring determinants of voting for the "helpfulness" of online user reviews: A text mining approach. Decision Support Systems, 50(2), 511-521. https://doi.org/10.1016/j.dss.2010.11.009 
  10. Chen, S. Y., Hsu, C. C., Kuo, C. C., and Ku, L. W. (2018). Emotionlines: An emotion corpus of multi-party conversations. arXiv preprint. arXiv: 1802.08379. 
  11. Chua, A. Y., and Banerjee, S. (2016). Helpfulness of user-generated reviews as a function of review sentiment, product type and information quality. Computers in Human Behavior, 54, 547-554. https://doi.org/10.1016/j.chb.2015.08.057 
  12. Clark, K., Luong, M. T., Le, Q. V., and Manning, C. D. (2020). ELECTRA: Pre-training text encoders as discriminators rather than generators. arXiv preprint. arXiv:2003.10555. 
  13. Craciun, G., and Moore, K. (2019). Credibility of negative online product reviews: Reviewer gender, reputation and emotion effects. Computers in Human Behavior, 97, 104-115. https://doi.org/10.1016/j.chb.2019.03.010 
  14. Craciun, G., Zhou, W., and Shan, Z. (2020). Discrete emotions effects on electronic word-of-mouth helpfulness: The moderating role of reviewer gender and contextual emotional tone. Decision Support Systems, 130, 113226. https://doi.org/10.1016/j.dss.2019.113226 
  15. Devlin, J., Chang, M. W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint. arXiv:1810.04805. 
  16. Dillard, J. P., Plotnick, C. A., Godbold, L. C., Freimuth, V. S., and Edgar, T. (1996). The multiple affective outcomes of AIDS PSAs: Fear appeals do more than scare people. Communication Research, 23(1), 44-72. https://doi.org/10.1177/009365096023001002 
  17. Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3-4), 169-200. https://doi.org/10.1080/02699939208411068 
  18. Fan, W. (2021). What makes consumer perception of online review helpfulness: synthesizing the past to guide future research. In Proceedings of the 54th Hawaii International Conference on System Sciences, 2738. 
  19. Ferrer, R., Klein, W., Lerner, J., Reyna, V., and Keltner, D. (2016). Emotions and health decision making. In C. Roberto and I. Kawachi (Eds.), Behavioral economics and Public Health (pp. 101-132). Cambridge, MA: Harvard University Press. 
  20. Filieri, R. (2015). What makes online reviews helpful? A diagnosticity-adoption framework to explain informational and normative influences in e-WOM. Journal of Business Research, 68(6), 1261-1270. https://doi.org/10.1016/j.jbusres.2014.11.006 
  21. Filieri, R., Galati, F., and Raguseo, E. (2021). The impact of service attributes and category on eWOM helpfulness: An investigation of extremely negative and positive ratings using latent semantic analytics and regression analysis. Computers in Human Behavior, 114, 106527. https://doi.org/10.1016/j.chb.2020.106527 
  22. Filieri, R., Raguseo, E., and Vitari, C. (2018). When are extreme ratings more helpful? Empirical evidence on the moderating effects of review characteristics and product type. Computers in Human Behavior, 88, 134-142. https://doi.org/10.1016/j.chb.2018.05.042 
  23. Fontaine, J. R., Scherer, K. R., Roesch, E. B., and Ellsworth, P. C. (2007). The world of emotions is not two-dimensional. Psychological science, 18(12), 1050-1057. https://doi.org/10.1111/j.1467-9280.2007.02024.x 
  24. Galati, F., and Galati, R. (2019). Cross-country analysis of perception and emphasis of hotel attributes. Tourism Management, 74, 24-42. https://doi.org/10.1016/j.tourman.2019.02.011 
  25. Garcia, D., and Schweitzer, F. (2011). Emotions in product reviews--empirics and models. In 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing (pp. 483-488). 
  26. Ghose, A., and Ipeirotis, P. G. (2010). Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics. IEEE Transactions on Knowledge and Data Engineering, 23(10), 1498-1512. https://doi.org/10.1109/TKDE.2010.188 
  27. Hardeniya, T., and Borikar, D. A. (2016). Dictionary based approach to sentiment analysis-a review. International Journal of Advanced Engineering, Management and Science, 2(5), 239438. 
  28. Hu, T., Wang, S., Luo, W., Zhang, M., Huang, X., Yan, Y., Liu, R., Ly, K., Kacker, V., She, B., and Li, Z. (2021). Revealing Public Opinion Towards COVID-19 Vaccines With Twitter Data in the United States: Spatiotemporal Perspective. Journal of Medical Internet Research, 23(9), e30854. https://doi.org/10.2196/30854 
  29. Huang, A. H., Chen, K., Yen, D. C., and Tran, T. P. (2015). A study of factors that contribute to online review helpfulness. Computers in Human Behavior, 48, 17-27. https://doi.org/10.1016/j.chb.2015.01.010 
  30. Janiesch, C., Zschech, P., and Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets, 31(3), 685-695.  https://doi.org/10.1007/s12525-021-00475-2
  31. Jiang, Z., and Benbasat, I. (2007). The effects of presentation formats and task complexity on online consumers' product understanding. Mis Quarterly, 475-500. 
  32. Karimi, S., and Wang, F. (2017). Online review helpfulness: Impact of reviewer profile image. Decision Support Systems, 96, 39-48. https://doi.org/10.1016/j.dss.2017.02.001 
  33. Khotimah, D. A. K., and Sarno, R. (2019). Sentiment analysis of hotel aspect using probabilistic latent semantic analysis, word embedding and LSTM. International Journal of Intelligent Engineering and Systems, 12(4), 275-290.  https://doi.org/10.22266/ijies2019.0831.26
  34. Kim, J., and Gupta, P. (2012). Emotional expressions in online user reviews: How they influence consumers' product evaluations. Journal of Business Research, 65(7), 985-992. https://doi.org/10.1016/j.jbusres.2011.04.013 
  35. Kim, T., and Vossen, P. (2021). EmoBERTa: Speaker-Aware Emotion Recognition in Conversation with RoBERTa. arXiv preprint. arXiv:2108.12009. 
  36. Kim, W. G., Li, J. J., and Brymer, R. A. (2016). The impact of social media reviews on restaurant performance: The moderating role of excellence certificate. International Journal of Hospitality Management, 55, 41-51. https://doi.org/10.1016/j.ijhm.2016.03.001 
  37. Lazarus, R. S. (1991). Emotion and Adaptation. Oxford University Press. 
  38. Lee, M., Jeong, M., and Lee, J. (2017). Roles of negative emotions in customers' perceived helpfulness of hotel reviews on a user-generated review website: A text mining approach. International Journal of Contemporary Hospitality Management, 29(2), 762-783. https://doi.org/10.1108/IJCHM-10-2015-0626 
  39. Lerner, J. S., and Keltner, D. (2001). Fear, anger, and risk. Journal of Personality and Social Psychology, 81(1), 146. https://doi.org/10.1037/0022-3514.81.1.146 
  40. Li, H., Liu, H., and Zhang, Z. (2020). Online persuasion of review emotional intensity: A text mining analysis of restaurant reviews. International Journal of Hospitality Management, 89, 102558. https://doi.org/10.1016/j.ijhm.2020.102558 
  41. Li, Y., Su, H., Shen, X., Li, W., Cao, Z., and Niu, S. (2017). DailyDialog: A manually labelled multi-turn dialogue dataset. arXiv preprint. arXiv:1710.03957. 
  42. Lin, T. M., Lu, K. Y., and Wu, J. J. (2012). The effects of visual information in eWOM communication. The Journal of Research in Indian Medicine, 6(1), 7-26. https://doi.org/10.1108/17505931211241341 
  43. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint. arXiv:1907.11692. 
  44. Liu, Z., and Park, S. (2015). What makes a useful online review? Implication for travel product websites. Tourism Management, 47, 140-151. https://doi.org/10.1016/j.tourman.2014.09.020 
  45. Ludwig, S., De Ruyter, K., Friedman, M., Bruggen, E. C., Wetzels, M., and Pfann, G. (2013). More than words: The influence of affective content and linguistic style matches in online reviews on conversion rates. Journal of Marketing, 77(1), 87-103. https://doi.org/10.1509/jm.11.0560 
  46. Ma, Y., Xiang, Z., Du, Q., and Fan, W. (2018). Effects of user-provided photos on hotel review helpfulness: An analytical approach with deep leaning. International Journal of Hospitality Management, 71, 120-131. https://doi.org/10.1016/j.ijhm.2017.12.008 
  47. Mohammad, S., and Turney, P. (2010). Emotions evoked by common words and phrases: Using mechanical turk to create an emotion lexicon. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text (pp. 26-34). 
  48. Mudambi, S. M., and Schuff, D. (2010). What makes a helpful review? A study of customer reviews on amazon. Com. MIS Quarterly, 34(1), 185-200.  https://doi.org/10.2307/20721420
  49. Nabi, R. (2002). Anger, fear, uncertainty, and attitudes: A test of the cognitive-functional model. Communication Monographs, 69(3), 204-216. https://doi.org/10.1080/03637750216541 
  50. Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17(4), 460-469. https://doi.org/10.1177/002224378001700405 
  51. Otterbacher, J. (2009). 'Helpfulness' in online communities: a measure of message quality. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 955-964). 
  52. Pan, Y., and Zhang, J. Q. (2011). Born unequal: A study of the helpfulness of user-generated product reviews. Journal of Retailing, 87(4), 598-612. https://doi.org/10.1016/j.jretai.2011.05.002 
  53. Park, D. H., and Lee, J. (2008). EWOM overload and its effect on consumer behavioral intention depending on consumer involvement. Electronic Commerce Research and Applications, 7(4), 386-398. https://doi.org/10.1016/j.elerap.2007.11.004 
  54. Peng, C., Yin, D., Wei, C., and Zhang, H. (2014). How and when review length and emotional intensity influence review helpfulness: Empirical evidence from epinions.com. In Proceedings of Thirty Fifth International Conference on Information Systems (pp. 1-16). 
  55. Pennebaker, J. W., Boyd, R. L., Jordan, K., and Blackburn, K. (2015). Linguistic Inquiry and WORD COUNT: LIWC2015. Austin, TX: Pennebaker Conglomerates. 
  56. Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., and Mihalcea, R. (2018). MELD: A multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint. arXiv: 1810.02508. 
  57. Provost, F., and Kohavi, R. (1998). Glossary of terms. Journal of Machine Learning, 30(2-3), 271-274. https://doi.org/10.1023/A:1017181826899 
  58. Qazi, A., Syed, K. B. S., Raj, R. G., Cambria, E., Tahir, M., and Alghazzawi, D. (2016). A concept-level approach to the analysis of online review helpfulness. Computers in Human Behavior, 58, 75-81. https://doi.org/10.1016/j.chb.2015.12.028 
  59. Racherla, P., and Friske, W. (2012). Perceived 'usefulness' of online consumer reviews: An exploratory investigation across three services categories. Electronic Commerce Research and Applications, 11(6), 548-559. https://doi.org/10.1016/j.elerap.2012.06.003 
  60. Ren, G., and Hong, T. (2019). Examining the relationship between specific negative emotions and the perceived helpfulness of online reviews. Information Processing and Management, 56(4), 1425-1438. https://doi.org/10.1016/j.ipm.2018.04.003 
  61. Rhee, H. T., and Yang, S. B. (2015). Does hotel attribute importance differ by hotel? Focusing on hotel star-classifications and customers' overall ratings. Computers in Human Behavior, 50, 576-587. https://doi.org/10.1016/j.chb.2015.02.069 
  62. Rozin, P., and Royzman, E. B. (2001). Negativity bias, negativity dominance, and contagion. Personality and Social Psychology Review, 5(4), 296-320. https://doi.org/10.1207/S15327957PSPR0504_2 
  63. Salehan, M., and Kim, D. J. (2016). Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics. Decision Support Systems, 81, 30-40. https://doi.org/10.1016/j.dss.2015.10.006 
  64. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint. arXiv: 1910.01108. 
  65. Shah, A. M., and Lee, K. (2022). The role of emotions intensity in helpfulness of online physician reviews. Intelligent Automation and Soft Computing, 31(3), 1719-1735. https://doi.org/10.32604/iasc.2022.019666 
  66. Silva, R. (2015). Multimarket contact, differentiation, and prices of chain hotels. Tourism Management, 48, 305-315. https://doi.org/10.1016/j.tourman.2014.11.006 
  67. Simons, T., and Hinkin, T. (2001). The effect of employee turnover on hotel profits: A test across multiple hotels. Cornell Hotel and Restaurant Administration Quarterly, 42(4), 65-69. https://doi.org/10.1016/S0010-8804(01)80046-X 
  68. Smith, C. A., and Ellsworth, P. C. (1985). Patterns of cognitive appraisal in emotion. Journal of ,ersonality and Social Psychology, 48(4), 813-838. https://doi.org/10.1037/0022-3514.48.4.813 
  69. Sniezek, J. A., and Van Swol, L. M. (2001). Trust, confidence, and expertise in a judge-advisor system. Organizational Behavior and Human Decision Processes, 84(2), 288-307. https://doi.org/10.1006/obhd.2000.2926 
  70. Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., and Liu, C. (2018). A survey on deep transfer learning. International Conference on Artificial Neural Networks, 270-279. 
  71. Tiedens, L. Z., and Linton, S. (2001). Judgment under emotional certainty and uncertainty: The effects of specific emotions on information processing. Journal of Personality and Social Psychology, 81(6), 973-988. https://doi.org/10.1037/0022-3514.81.6.973 
  72. Veall, M. R., and Zimmermann, K. F. (1996). Pseudo-R2 measures for some common limited dependent variable models. Journal of Economic Surveys, 10(3), 241-259. https://doi.org/10.1111/j.1467-6419.1996.tb00013.x 
  73. Wang, X., Tang, L. R., and Kim, E. (2019). More than words: Do emotional content and linguistic style matching matter on restaurant review helpfulness? International Journal of Hospitality Management, 77, 438-447. https://doi.org/10.1016/j.ijhm.2018.08.007 
  74. Weiss, A. M., Lurie, N. H., and MacInnis, D. J. (2008). Listening to strangers: whose responses are valuable, how valuable are they, and why? Journal of marketing Research, 45(4), 425-436. https://doi.org/10.1509/jmkr.45.4.425 
  75. Westbrook, R. A., and Oliver, R. L. (1991). The dimensionality of consumption emotion patterns and consumer satisfaction. Journal of Consumer Research, 18(1), 84-91. https://doi.org/10.1086/209243 
  76. Xie, H. J., Miao, L., Kuo, P. J., and Lee, B. Y. (2011). Consumers' responses to ambivalent online hotel reviews: The role of perceived source credibility and pre-decisional disposition. International Journal of Hospitality Management, 30(1), 178-183. https://doi.org/10.1016/j.ijhm.2010.04.008 
  77. Yan, X., Khan, S., and Shah, S. J. (2020). Exploring the impact of review and service-related signals on online physician review helpfulness: A multi-methods approach. In Proceedings of Twenty-Fourth Pacific Asia Conference on Information Systems (pp. 1-14). 
  78. Yang, S., Zhou, C., and Chen, Y. (2021). Do topic consistency and linguistic style similarity affect online review helpfulness? An elaboration likelihood model perspective. Information Processing and Management, 58(3), 102521. https://doi.org/10.1016/j.ipm.2021.102521 
  79. Yang, S., Yao, J., and Qazi, A. (2020). Does the review deserve more helpfulness when its title resembles the content? Locating helpful reviews by text mining. Information Processing and Management, 57(2), 102179. https://doi.org/10.1016/j.ipm.2019.102179 
  80. Yin, D., Bond, S. D., and Zhang, H. (2014). Anxious or angry? Effects of discrete emotions on the perceived helpfulness of online reviews. MIS Quarterly, 38(2), 539-560.  https://doi.org/10.25300/MISQ/2014/38.2.10
  81. Yin, D., Bond, S. D., and Zhang, H. (2017). Keep your cool or let it out: Nonlinear effects of expressed arousal on perceptions of consumer reviews. Journal of Marketing Research, 54(3), 447-463. https://doi.org/10.1509/jmr.13.0379 
  82. Zhou, S., and Guo, B. (2017). The order effect on online review helpfulness: A social influence perspective. Decision Support Systems, 93, 77-87. https://doi.org/10.1016/j.dss.2016.09.016 
  83. Zhu, L., Yin, G., and He, W. (2014). Is this opinion leader's review useful? Peripheral cues for online review helpfulness. Journal of Electronic Commerce Research, 15(4), 267-280.