Asia pacific journal of information systems

The Korea Society of Management Information Systems (한국경영정보학회)
권호 표지
DOI QR코드

DOI QR Code

Assessing Personalized Recommendation Services Using Expectancy Disconfirmation Theory

  • Il Young Choi (Graduate School of Business Administration & AI Research Center, Kyung Hee University) ;
  • Hyun Sil Moon (Graduate School of Business Administration & AI Research Center, Kyung Hee University) ;
  • Jae Kyeong Kim (School of Management, Kyung Hee University & AI Research Center, Kyung Hee University)
  • Received : 2019.01.14
  • Accepted : 2019.04.23
  • Published : 2019.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

There is an accuracy-diversity dilemma with personalized recommendation services. Some researchers believe that accurate recommendations might reinforce customer satisfaction. However, others claim that highly accurate recommendations and customer satisfaction are not always correlated. Thus, this study attempts to establish the causal factors that determine customer satisfaction with personalized recommendation services to reconcile these incompatible views. This paper employs statistical analyses of simulation to investigate an accuracy-diversity dilemma with personalized recommendation services. To this end, we develop a personalized recommendation system and measured accuracy, diversity, and customer satisfaction using a simulation method. The results show that accurate recommendations positively affected customer satisfaction, whereas diverse recommendations negatively affected customer satisfaction. Also, customer satisfaction was associated with the recommendation product size when neighborhood size was optimal in accuracy. Thus, these results offer insights into personalizing recommendation service providers. The providers must identify customers' preferences correctly and suggest more accurate recommendations. Furthermore, accuracy is not always improved as the number of product recommendation increases. Accordingly, providers must propose adequate number of product recommendation.

Keywords

References

  1. Abdel-Hafez, A., Tang, X., Tian, N., and Xu, Y. (2014). A reputation-enhanced recommender system. Advanced Data Mining and Applications, Springer, Cham.
  2. Adomavicius, G., and Kwon, Y. (2008). Overcoming accuracy-diversity tradeoff in recommender systems: a variance-based approach. In Proceedings of the 18th Workshop on Information Technology and Systems.
  3. Adomavicius, G., and Kwon, Y. (2012). Improving aggregate recommendation diversity using ranking-based techniques. IEEE Transactions on Knowledge and Data Engineering, 24(5), 896-911. https://doi.org/10.1109/TKDE.2011.15
  4. Ahn, H., Han, I., and Kim, K. J. (2006). The product recommender system combining association rules and classification models: the case of g internet shopping mall. Information Systems Review, 8(1), 181-201.
  5. Athiyaman, A. (1997). Linking student satisfaction and service quality perceptions: the case of university education. European Journal of Marketing, 31(7), 528-540. https://doi.org/10.1108/03090569710176655
  6. Bennett, J., and Lanning, S. (2007). The Netflix prize. In Proceedings of KDD Cup and Workshop.
  7. Bhattacherjee, A. (2001). Understanding information systems continuance: an expectation-confirmation model. MIS Quarterly, 25(3), 351-370. https://doi.org/10.2307/3250921
  8. Bitner, M. J. (1990). Evaluating service encounters: the effects of physical surroundings and employee responses. Journal of Marketing, 54(2), 69-82. https://doi.org/10.1177/002224299005400206
  9. Calvo-Porral, C., and Levy-Mangin, J. (2015). Switching behavior and customer satisfaction in mobile services: analyzing virtual and traditional operators. Computers in Human Behavior, 49, 532-540. https://doi.org/10.1016/j.chb.2015.03.057
  10. Chandra, A., Chen, H., and Yao, X. (2006). Trade-off between diversity and accuracy in ensemble generation. Multi-Objective Machine Learning, Springer Verlag, Berlin.
  11. Chen, D. N., Hu, P. J. H., Kuo, Y. R., and Liang, T. P. (2010). A Web-based personalized recommendation system for mobile phone selection: Design, implementation, and evaluation. Expert Systems with Applications, 37(12), 8201-8210. https://doi.org/10.1016/j.eswa.2010.05.066
  12. Cho, Y. H., and Kim, J. K. (2004). Application of web usage mining and product taxonomy to collaborative recommendations in e-commerce. Expert Systems with Applications, 26(2), 233-246. https://doi.org/10.1016/S0957-4174(03)00138-6
  13. Cho, Y. H., Kim, J. K., and Kim, S. H. (2002). A personalized recommender system based on web usage mining and decision tree induction. Expert Systems with Applications, 23(3), 329-342. https://doi.org/10.1016/S0957-4174(02)00052-0
  14. Chong, B., and Wong, M. (2005). Crafting an effective customer retention strategy: a review of halo effect on customer satisfaction in online auctions. International Journal of Management and Enterprise Development, 2(1), 12-26. https://doi.org/10.1504/IJMED.2005.006021
  15. Christoffel, F., Paudel, B., Newell, C., and Bernstein, A. (2015). Blockbusters and wallflowers: accurate, diverse, and scalable recommendations with random walks. In Proceedings of the 9th ACM Conference on Recommender Systems, ACM, 163-170.
  16. Cremonesi, P., Garzotto, F., Negro, S., Papadopoulos, A. V., and Turrin, R. (2011). Looking for "good" recommendations: a comparative evaluation of recommender systems. Human-Computer Interaction-INTERACT, Springer Verlag, Berlin.
  17. Das, A. S., Datar, M., Garg, A., and Rajaram, S. (2007). Google news personalization: scalable online collaborative filtering. In Proceedings of the 16th International Conference on World Wide Web, ACM, 271-280.
  18. Deng, Z., Lu, Y., Wei, K. K., and Zhang, J. (2010). Understanding customer satisfaction and loyalty: an empirical study of mobile instant messages in China. International Journal of Information Management, 30(4), 289-300. https://doi.org/10.1016/j.ijinfomgt.2009.10.001
  19. Ekstrand, M. D., Harper, F. M., Willemsen, M. C., and Konstan, J. A. (2014). User perception of differences in recommender algorithms. In Proceedings of the 8th ACM Conference on Recommender Systems, ACM, 161-168.
  20. Fitzsimons, G. J., and Lehmann, D. R. (2004). Reactance to recommendations: when unsolicited advice yields contrary responses. Marketing Science, 23(1), 82-94. https://doi.org/10.1287/mksc.1030.0033
  21. Gan, M., and Jiang, R. (2013). Constructing a user similarity network to remove adverse influence of popular objects for personalized recommendation. Expert Systems with Applications, 40(10), 4044-4053. https://doi.org/10.1016/j.eswa.2013.01.004
  22. Gerpott, T. J., Rams, W., and Schindler, A. (2001). Customer retention, loyalty, and satisfaction in the German mobile cellular telecommunications market, Telecomm. Policy, 25, 249-269. https://doi.org/10.1016/S0308-5961(00)00097-5
  23. Herlocker, J. L., Konstan, J. A., and Riedl, J. (2000). Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, ACM, 241-250.
  24. Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. (2004). Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS), 22(1), 5-53. https://doi.org/10.1145/963770.963772
  25. Hijikata, Y., Shimizu, T., and Nishida, S. (2009). Discovery-oriented collaborative filtering for improving user satisfaction. In Proceedings of the 14th International Conference on Intelligent User Interfaces, ACM, 67-76.
  26. Hill, F. M. (1995). Managing service quality in higher education: the role of the student as primary consumer. Quality Assurance in Education, 3(3), 10-21. https://doi.org/10.1108/09684889510093497
  27. Hill, W., Stead, L., Rosenstein, M., and Furnas, G. (1995). Recommending and evaluating choices in a virtual community of use. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., 194-201.
  28. Im, I., and Hars, A. (2007). Does a one-size recommendation system fit all? the effectiveness of collaborative filtering based recommendation systems across different domains and search modes. ACM Transactions on Information Systems, 26(1), 4.
  29. Javari, A., and Jalili, M. (2015). A probabilistic model to resolve diversity-accuracy challenge of recommendation systems. Knowledge and Information Systems, 44(3), 609-627. https://doi.org/10.1007/s10115-014-0779-2
  30. Jiang, Y., Shang, J., and Liu, Y. (2010). Maximizing customer satisfaction through an online recommendation system: a novel associative classification model. Decision Support Systems, 48(3), 470-479. https://doi.org/10.1016/j.dss.2009.06.006
  31. Kaminskas, M., and Bridge, D. (2017). Diversity, serendipity, novelty, and coverage: A survey and empirical analysis of beyond-accuracy objectives in recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS), 7(1), 2.
  32. Kim, H. K., Kim, J. K., and Ryu, Y. U. (2009). Personalized recommendation over a customer network for ubiquitous shopping. IEEE Transactions on Services Computing, 2(2), 140-151. https://doi.org/10.1109/TSC.2009.7
  33. Kim, H., Ji, A., Ha, I., and Jo, G. (2010). Collaborative filtering based on collaborative tagging for enhancing the quality of recommendation. Electronic Commerce Research and Applications, 9(1), 73-83. https://doi.org/10.1016/j.elerap.2009.08.004
  34. Konstan, J. A., Miller, B. N., Maltz, D., Herlocker, J. L., Gordon, L. R., and Riedl, J. (1997). GroupLens: applying collaborative filtering to Usenet news. Communications of the ACM, 40(3), 77-87. https://doi.org/10.1145/245108.245126
  35. Koren, Y. (2010). Factor in the neighbors. ACM Transactions on Knowledge Discovery from Data, 4(1), 1-24. https://doi.org/10.1145/1644873.1644874
  36. Lawrence, R. D., Almasi, G. S., Kotlyar, V., Viveros, M., and Duri, S. S. (2001). Personalization of supermarket product recommendations. Applications of Data Mining to Electronic Commerce, Springer, New York.
  37. Lee, K., and Lee, K. (2015). Escaping your comfort zone: a graph-based recommender system for finding novel recommendations among relevant items. Expert Systems with Applications, 42(10), 4851-4858. https://doi.org/10.1016/j.eswa.2014.07.024
  38. Liang, T., Lai, H., and Ku, Y. (2007). Personalized content recommendation and user satisfaction: theoretical synthesis and empirical findings. Journal of Management Information Systems, 23(3), 45-70. https://doi.org/10.2753/MIS0742-1222230303
  39. Linden, G., Smith, B., and York, J. (2003). Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Computing, 7(1), 76-80. https://doi.org/10.1109/MIC.2003.1167344
  40. Liu, H., Hu, Z., Mian, A., Tian, H., and Zhu, X. (2014). A new user similarity model to improve the accuracy of collaborative filtering. Knowledge-Based Systems, 56, 156-166. https://doi.org/10.1016/j.knosys.2013.11.006
  41. Maxham, J. G. (2001). Service recovery's influence on consumer satisfaction, positive word-of-mouth, and purchase intentions. Journal of Business Research, 54(1), 11-24. https://doi.org/10.1016/S0148-2963(00)00114-4
  42. McGinty, L., and Smyth, B. (2003). On the role of diversity in conversational recommender systems. In International Conference on Case-Based Reasoning (pp. 276-290). Springer, Berlin, Heidelberg.
  43. McNee, S. M., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S. K., Rashid, A. M., Konstan, J. A., and Riedl, J. (2002). On the recommending of citations for research papers. In Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work, ACM, 116-125.
  44. McNee, S. M., Riedl, J., and Konstan, J. A. (2006). Being accurate is not enough: how accuracy metrics have hurt recommender systems. In CHI'06 extended abstracts on Human factors in computing systems (pp. 1097-1101). Acm.
  45. Moon, H. S., Kim, J. K., and Ryu, Y. U. (2013). A sequence-based filtering method for exhibition booth visit recommendations. International Journal of Information Management, 33(4), 620-626. https://doi.org/10.1016/j.ijinfomgt.2013.03.004
  46. Moon, H. S., Yoon, J. H., Choi, I. Y., and Kim, J. K. (2017). An exploratory study of collaborative filtering techniques to analyze the effect of information amount. Asia Pacific Journal of Information Systems, 27(2), 126-138. https://doi.org/10.14329/apjis.2017.27.2.126
  47. Mudambi, S. M., and Schuff, D. (2010). What makes a helpful review? A study of customer reviews on Amazon.com. MIS Quarterly, 34, 185-200. https://doi.org/10.2307/20721420
  48. Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17(4), 460-469. https://doi.org/10.1177/002224378001700405
  49. Pu, P., Chen, L., and Hu, R. (2011). A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems (pp. 157-164). ACM.
  50. Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., and Riedl, J. (1994). GroupLens: an open architecture for collaborative filtering of netnews. In Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work, ACM, 175-186.
  51. Roca, J. C., Chiu, C., and Martinez, F. J. (2006). Understanding e-learning continuance intention: an extension of the technology acceptance model. International Journal of Human-Computer Studies, 64(8), 683-696. https://doi.org/10.1016/j.ijhcs.2006.01.003
  52. Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. (2000). Analysis of recommendation algorithms for e-commerce. In Proceedings of the 2nd ACM Conference on Electronic Commerce, ACM, 158-167.
  53. Shardanand, U., and Maes, P. (1995). Social information filtering: algorithms for automating "word of mouth". In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM Press/Addison-Wesley Publishing Co., 210-217.
  54. Smyth, B., and McClave, P. (2001). Similarity vs. diversity. Case-Based Reasoning Research and Development, Springer Verlag, Berlin.
  55. Sohn, J., and Suh, Y. M. (2006). Using degree of match to improve prediction quality in collaborative filtering systems. Information Systems Review, 8(2), 139-154.
  56. Suh, K. S., Lee, S., Suh, E. K., Kang, H., Lee, S., and Lee, U. K. (2014). Comparisons of popularity-and expert-based news recommendations: similarities and importance. Asia Pacific Journal of Information Systems, 24(2), 191-210. https://doi.org/10.14329/apjis.2014.24.2.191
  57. Thongpapanl, N., and Ashraf, A. R. (2011). Enhancing online performance through website content and personalization. Journal of Computer Information Systems, 52(1), 3-13.
  58. Tsai, C., and Hung, C. (2012). Cluster ensembles in collaborative filtering recommendation. Applied Soft Computing, 12(4), 1417-1425. https://doi.org/10.1016/j.asoc.2011.11.016
  59. Willemsen, M. C., Knijnenburg, B. P., Graus, M. P., Velter-Bremmers, L. C., and Fu, K. (2011). Using latent features diversification to reduce choice difficulty in recommendation lists. RecSys 2011, 14-20.
  60. Yu, K., Schwaighofer, A., Tresp, V., Xu, X., and Kriegel, H. (2004). Probabilistic memory-based collaborative filtering. IEEE Transactions on Knowledge and Data Engineering, 16(1), 56-69. https://doi.org/10.1109/TKDE.2004.1264822
  61. Zhao, L., and Lu, Y. (2012). Enhancing perceived interactivity through network externalities: an empirical study on micro-blogging service satisfaction and continuance intention. Decision Support Systems, 53(4), 825-834. https://doi.org/10.1016/j.dss.2012.05.019
  62. Zheng, N., Li, Q., Liao, S., and Zhang, L. (2010). Which photo groups should I choose? A comparative study of recommendation algorithms in Flickr. Journal of Information Science, 36(6), 733-750. https://doi.org/10.1177/0165551510386164
  63. Zhou, T., Kuscsik, Z., Liu, J. G., Medo, M., Wakeling, J. R., and Zhang, Y. C. (2010). Solving the apparent diversity-accuracy dilemma of recommender systems. Proceedings of the National Academy of Sciences of the United States of America, 107(10), 4511-4515. https://doi.org/10.1073/pnas.1000488107
  64. Zhou, X., Xu, Y., Li, Y., Josang, A., and Cox, C. (2012). The state-of-the-art in personalized recommender systems for social networking. Artificial Intelligence Review, 37(2), 119-132. https://doi.org/10.1007/s10462-011-9222-1
  65. Ziegler, C. N., McNee, S. M., Konstan, J. A., and Lausen, G. (2005). Improving recommendation lists through topic diversification. In Proceedings of the 14th International Conference on World Wide Web, ACM, 22-32.
  66. Zins, A. H., Bauemfeind, U., Missier, F., Venturini, A., and Rumetshofer, H. (2004). An experimental usability test for different destination recommender systems. Information and Communication Technologies in Tourism, Springer Verlag, New York.