Asia pacific journal of information systems

  • 제29권4호
  • /
  • Pages.856-874
  • /
  • 2019
  • /
  • 2288-5404(pISSN)
  • /
  • 2288-6818(eISSN)
한국경영정보학회 (The Korea Society of Management Information Systems)
권호 표지
DOI QR코드

DOI QR Code

The Effects of Content and Distribution of Recommended Items on User Satisfaction: Focus on YouTube

  • 투고 : 2019.05.11
  • 심사 : 2019.09.25
  • 발행 : 2019.12.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

The performance of recommender systems (RS) has been measured mainly in terms of accuracy. However, there are other aspects of performance that are difficult to understand in terms of accuracy, such as coverage, serendipity, and satisfaction with recommended results. Moreover, particularly with RSs that suggest multiple items at a time, such as YouTube, user satisfaction with recommended results may vary not only depending on their accuracy, but also on their configuration, content, and design displayed to the user. This is true when classifying an RS as a single RS with one recommended result and as a multiple RS with diverse results. No empirical analysis has been conducted on the influence of the content and distribution of recommendation items on user satisfaction. In this study, we propose a research model representing the content and distribution of recommended items and how they affect user satisfaction with the RS. We focus on RSs that recommend multiple items. We performed an empirical analysis involving 149 YouTube users. The results suggest that user satisfaction with recommended results is significantly affected according to the HHI (Herfindahl-Hirschman Index). In addition, satisfaction significantly increased when the recommended item on the top of the list was the same category in terms of content that users were currently watching. Particularly when the purpose of using RS is hedonic, not utilitarian, the results showed greater satisfaction when the number of views of the recommended items was evenly distributed. However, other characteristics of selected content, such as view count and playback time, had relatively less impact on satisfaction with recommended items. To the best of our knowledge, this study is the first to show that the category concentration of items impacts user satisfaction on websites recommending diverse items in different categories using a content-based filtering system, such as YouTube. In addition, our use of the HHI index, which has been extensively used in economics research, to show the distributional characteristics of recommended items, is also unique. The HHI for categories of recommended items was useful in explaining user satisfaction.

키워드

과제정보

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2017S1A3A2066740).

참고문헌

  1. Abbas, M., Riaz, M. U., Rauf, A., Khan, M. T., and Khalid, S. (2017, December). Context-aware Youtube recommender system. In 2017 International Conference on Information and Communication Technologies (ICICT) (pp. 161-164). IEEE. 
  2. Adamopoulos, P., and Tuzhilin, A. (2015). On unexpectedness in recommender systems: Or how to better expect the unexpected. ACM Transactions on Intelligent Systems and Technology (TIST), 5(4), 54. 
  3. Adomavicius, G., and Kwon, Y. (2011, October). Maximizing aggregate recommendation diversity: A graph-theoretic approach. In Proc. of the 1st International Workshop on Novelty and Diversity in Recommender Systems (DiveRS 2011) (pp. 3-10). 
  4. Anand, D., and Bharadwaj, K. K. (2011). Utilizing various sparsity measures for enhancing accuracy of collaborative recommender systems based on local and global similarities. Expert systems with applications, 38(5), 5101-5109.  https://doi.org/10.1016/j.eswa.2010.09.141
  5. Anderson, C. (2006). The long tail: Why the future of business is selling less of more. Hachette Books. 
  6. Baier, D., and Stuber, E. (2010). Acceptance of recommendations to buy in online retailing. Journal of Retailing and Consumer Services, 17(3), 173-180.  https://doi.org/10.1016/j.jretconser.2010.03.005
  7. Bobadilla, J., Ortega, F., Hernando, A., and Gutierrez, A. (2013). Recommender systems survey. Knowledge-based systems, 46, 109-132.  https://doi.org/10.1016/j.knosys.2013.03.012
  8. Bremus, F., and Buch, C. M. (2015). Banking market structure and macroeconomic stability: Are low income countries special? Pacific Economic Review, 20(1), 73-100.  https://doi.org/10.1111/1468-0106.12095
  9. Choi, J., Lee, H. J., and Kim, Y. C. (2011). The influence of social presence on customer intention to reuse online recommender systems: The roles of personalization and product type. International Journal of Electronic Commerce, 16(1), 129-154.  https://doi.org/10.2753/JEC1086-4415160105
  10. Cooper, J., and Mackie, D. (1986). Video games and aggression in children 1. Journal of Applied Social Psychology, 16(8), 726-744.  https://doi.org/10.1111/j.1559-1816.1986.tb01755.x
  11. Covington, P., Adams, J., and Sargin, E. (2016, September). Deep neural networks for Youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems (pp. 191-198). ACM. 
  12. De, P., Hu, Y., and Rahman, M. S. (2010). Technology usage and online sales: An empirical study. Management Science, 56(11), 1930-1945.  https://doi.org/10.1287/mnsc.1100.1233
  13. Ferwerda, B., Graus, M. P., Vall, A., Tkalcic, M., and Schedl, M. (2016, August). The influence of users' personality traits on satisfaction and attractiveness of diversified recommendation lists. In EMPIRE@ RecSys (pp. 43-47). 
  14. Gatian, A. W. (1994). Is user satisfaction a valid measure of system effectiveness? Information & Management, 26(3), 119-131.  https://doi.org/10.1016/0378-7206(94)90036-1
  15. Ge, M., Delgado-Battenfeld, C., and Jannach, D. (2010, September). Beyond accuracy: Evaluating recommender systems by coverage and serendipity. In Proceedings of the fourth ACM conference on Recommender systems (pp. 257-260). ACM. 
  16. Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. (2004). Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS), 22(1), 5-53.  https://doi.org/10.1145/963770.963772
  17. Hirschman, E. C., and Holbrook, M. B. (1982). Hedonic consumption: emerging concepts, methods and propositions. Journal of marketing, 46(3), 92-101.  https://doi.org/10.1177/002224298204600314
  18. Hong, S. E., and Kim, H. J. (2016, July). A comparative study of video recommender systems in big data era. In 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN) (pp. 125-127). IEEE. 
  19. Huang, L. T. (2016). Exploring utilitarian and hedonic antecedents for adopting information from a recommendation agent and unplanned purchase behaviour. New Review of Hypermedia and Multimedia, 22(1-2), 139-165.  https://doi.org/10.1080/13614568.2015.1052098
  20. Iaquinta, L., De Gemmis, M., Lops, P., Semeraro, G., Filannino, M., and Molino, P. (2008, September). Introducing serendipity in a content-based recommender system. In 2008 Eighth International Conference on Hybrid Intelligent Systems (pp. 168-173). IEEE. 
  21. Iaquinta, L., de Gemmis, M., Lops, P., Semeraro, G., and Molino, P. (2010). Can a recommender system induce serendipitous encounters?. In E-commerce. IntechOpen. 
  22. Jalili, M., Ahmadian, S., Izadi, M., Moradi, P., and Salehi, M. (2018). Evaluating collaborative filtering recommender algorithms: a survey. IEEE Access, 6, 74003-74024.  https://doi.org/10.1109/ACCESS.2018.2883742
  23. Kaminskas, M., and Bridge, D. (2014, October). Measuring surprise in recommender systems. In Proceedings of the workshop on recommender systems evaluation: dimensions and design (Workshop programme of the 8th ACM conference on recommender systems). 
  24. Kaminskas, M., and Bridge, D. (2017). Diversity, serendipity, novelty, and coverage: a survey and empirical analysis of beyond-accuracy objectives in recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS), 7(1), 2. 
  25. Katz, M. L., and Shapiro, C. (1985). Network externalities, competition, and compatibility. American Economic review, 75(3), 424-440. 
  26. Kotkov, D., Konstan, J. A., Zhao, Q., and Veijalainen, J. (2018, April). Investigating serendipity in recommender systems based on real user feedback. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing (pp. 1341-1350). ACM. 
  27. Kotkov, D., Wang, S., and Veijalainen, J. (2016). A survey of serendipity in recommender systems. Knowledge-Based Systems, 111, 180-192.  https://doi.org/10.1016/j.knosys.2016.08.014
  28. Kumar, A., Gupta, S., Singh, S. K., and Shukla, K. K. (2015, August). Comparison of various metrics used in collaborative filtering for recommendation system. In 2015 Eighth International Conference on Contemporary Computing (IC3) (pp. 150-154). IEEE. 
  29. Kunaver, M., and Pozrl, T. (2017). Diversity in recommender systems:A survey. Knowledge-Based Systems, 123, 154-162.  https://doi.org/10.1016/j.knosys.2017.02.009
  30. Lee, D., and Hosanagar, K. (2016, April). When do recommender systems work the best?: The moderating effects of product attributes and consumer reviews on recommender performance. In Proceedings of the 25th International Conference on World Wide Web (pp. 85-97). International World Wide Web Conferences Steering Committee. 
  31. Lee, J. K., Wang, J., and Lee, W. (2012). How is strategic brand alliance communicated to consumers? A content assessment of print ads. The Journal of Advertising and Promotion Research, 1(2), 5-40.  https://doi.org/10.14377/JAPR.2012.9.30.5
  32. Li, C., and Zhou, H. (2018). Enhancing the efficiency of massive online learning by integrating intelligent analysis into MOOCs with an application to education of sustainability. Sustainability, 10(2), 468. 
  33. Li, W., Xu, G., Xing, Q., and Lyu, M. (2019). Effect of directors' social network centrality on corporate charitable donation. Social Behavior and Personality: An international journal, 47(3), 1-9.  https://doi.org/10.2224/sbp.7681
  34. Liang, T. P., Lai, H. J., and Ku, Y. C. (2006). Personalized content recommendation and user satisfaction: Theoretical synthesis and empirical findings. Journal of Management Information Systems, 23(3), 45-70.  https://doi.org/10.2753/MIS0742-1222230303
  35. Lin, C. P., and Bhattacherjee, A. (2008). Elucidating individual intention to use interactive information technologies: The role of network externalities. International Journal of Electronic Commerce, 13(1), 85-108.  https://doi.org/10.2753/JEC1086-4415130103
  36. Lin, K. Y., and Lu, H. P. (2011). Intention to continue using Facebook fan pages from the perspective of social capital theory. Cyberpsychology, Behavior, and Social Networking, 14(10), 565-570.  https://doi.org/10.1089/cyber.2010.0472
  37. Lu, J., Wu, D., Mao, M., Wang, W., and Zhang, G. (2015). Recommender system application developments: A survey. Decision Support Systems, 74, 12-32.  https://doi.org/10.1016/j.dss.2015.03.008
  38. Lu, L., Medo, M., Yeung, C. H., Zhang, Y. C., Zhang, Z. K., and Zhou, T. (2012). Recommender systems. Physics reports, 519(1), 1-49.  https://doi.org/10.1016/j.physrep.2012.02.006
  39. Lu, C., Qiao, J., and Chang, J. (2017). Herfindahl-Hirschman Index based performance analysis on the convergence development. Cluster computing, 20(1), 121-129.  https://doi.org/10.1007/s10586-017-0737-3
  40. McNee, S. M., Riedl, J., and Konstan, J. A. (2006, April). Being accurate is not enough: how accuracy metrics have hurt recommender systems. In CHI'06 extended abstracts on Human factors in computing systems (pp. 1097-1101). ACM. 
  41. Mehrotra, R., McInerney, J., Bouchard, H., Lalmas, M., and Diaz, F. (2018, October). Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (pp. 2243-2251). ACM. 
  42. Meseguer-Martinez, A., Ros-Galvez, A., and Rosa-Garcia, A. (2017). Satisfaction with online teaching videos: A quantitative approach. Innovations in Education and Teaching International 54(1), 62-67.  https://doi.org/10.1080/14703297.2016.1143859
  43. Moghavvemi, S., Sulaiman, A. B., Jaafar, N. I. B., and Kasem, N. (2017, July). Facebook and YouTube addiction: The usage pattern of Malaysian students. In 2017 international conference on research and innovation in information systems (ICRIIS) (pp. 1-6). IEEE. 
  44. Murakami, T., Mori, K., and Orihara, R. (2007, June). Metrics for evaluating the serendipity of recommendation lists. In Annual conference of the Japanese society for artificial intelligence (pp. 40-46). Springer, Berlin, Heidelberg. 
  45. Nakatsuji, M., Fujiwara, Y., Tanaka, A., Uchiyama, T., Fujimura, K., and Ishida, T. (2010, October). Classical music for rock fans?: novel recommendations for expanding user interests. In Proceedings of the 19th ACM international conference on Information and knowledge management (pp. 949-958). ACM. 
  46. Nguyen, T. T., Harper, F. M., Terveen, L., and Konstan, J. A. (2018). User personality and user satisfaction with recommender systems. Information Systems Frontiers, 20(6), 1173-1189.  https://doi.org/10.1007/s10796-017-9782-y
  47. Ogara, S. O., Koh, C. E., and Prybutok, V. R. (2014). Investigating factors affecting social presence and user satisfaction with mobile instant messaging. Computers in Human Behavior, 36, 453-459.  https://doi.org/10.1016/j.chb.2014.03.064
  48. Onuma, K., Tong, H., and Faloutsos, C. (2009, June). TANGENT: A novel, 'Surprise me', recommendation algorithm. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 657-666). ACM. 
  49. Qiao, Z., and Li, Z. (2018). Do foreign institutional investors enhance firm innovation in China? Applied Economics Letters, 1-4. 
  50. Qin, S., Menezes, R., and Silaghi, M. (2010, August). A recommender system for youtube based on its network of reviewers. In 2010 IEEE Second International Conference on Social Computing (pp. 323-328). IEEE. 
  51. Salter, J., and Antonopoulos, N. (2006). CinemaScreen recommender agent: Combining collaborative and content-based filtering. IEEE Intelligent Systems, 21(1), 35-41.  https://doi.org/10.1109/MIS.2006.4
  52. Shani, G., and Gunawardana, A. (2011). Evaluating recommendation systems. In Recommender systems handbook(pp. 257-297). Springer, Boston, MA. 
  53. Sridharan, S. (2014). Introducing serendipity in recommender systems through collaborative methods. Master's Theses. University of Rhode Island. 
  54. Strahilevitz, M., and Myers, J. G. (1998). Donations to charity as purchase incentives: How well they work may depend on what you are trying to sell. Journal of consumer research, 24(4), 434-446.  https://doi.org/10.1086/209519
  55. Tang, T. Y., and Winoto, P. (2016). I should not recommend it to you even if you will like it: The ethics of recommender systems. New Review of Hypermedia and Multimedia, 22(1-2), 111-138.  https://doi.org/10.1080/13614568.2015.1052099
  56. Thorat, P. B., Goudar, R. M., and Barve, S. (2015). Survey on collaborative filtering, content-based filtering and hybrid recommendation system. International Journal of Computer Applications, 110(4), 31-36.  https://doi.org/10.5120/19308-0760
  57. Vargas, S., and Castells, P. (2011, October). Rank and relevance in novelty and diversity metrics for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems (pp. 109-116). ACM. 
  58. Vargas, S., Baltrunas, L., Karatzoglou, A., and Castells, P. (2014, October). Coverage, redundancy and size-awareness in genre diversity for recommender systems. In Proceedings of the 8th ACM Conference on Recommender systems (pp. 209-216). ACM. 
  59. Wallner, G., Schinnerl, C., Schiller, M. H., Calvo, A. M., Pirker, J., Sifa, R., and Drachen, A. (2019). Beyond the individual: Understanding social structures of an online player matchmaking website. Entertainment Computing, 30, 100284. 
  60. Wang, E. S. T., and Chen, L. S. L. (2012). Forming relationship commitments to online communities: The role of social motivations. Computers in Human Behavior, 28(2), 570-575.  https://doi.org/10.1016/j.chb.2011.11.002
  61. Yao, Y. Y. (1995). Measuring retrieval effectiveness based on user preference of documents. Journal of the American Society for Information Science, 46(2), 133-145.  https://doi.org/10.1002/(SICI)1097-4571(199503)46:2<133::AID-ASI6>3.0.CO;2-Z
  62. Yoo, W. S., Lee, Y., and Park, J. (2010). The role of interactivity in e-tailing: Creating value and increasing satisfaction. Journal of retailing and consumer services, 17(2), 89-96.  https://doi.org/10.1016/j.jretconser.2009.10.003
  63. Yoo, S. J. (2016). Reliability-based similarity evaluation model for collaborative filtering. Journal of Advanced Information Technology and Convergence, 14(12), 141-146. 
  64. Zeng, A., Yeung, C. H., Medo, M., and Zhang, Y. C. (2015). Modeling mutual feedback between users and recommender systems. Journal of Statistical Mechanics: Theory and Experiment, 7, 07020. 
  65. Zhang, S., Zhao, T., and Xie, B. C. (2018). What is the optimal power generation mix of China? An empirical analysis using portfolio theory. Applied energy, 229, 522-536.  https://doi.org/10.1016/j.apenergy.2018.08.028
  66. Zhang, W., Gu, T., Sun, W., Phatpicha, Y., Chang, L., and Bin, C. (2018, September). Travel attractions recommendation with travel spatial-temporal knowledge graphs. In International Conference of Pioneering Computer Scientists, Engineers and Educators (pp. 213-226). Springer, Singapore. 
  67. Zhang, Y. C., Seaghdha, D. O., Quercia, D., and Jambor, T. (2012, February). Auralist: Introducing serendipity into music recommendation. In Proceedings of the fifth ACM international conference on Web search and data mining (pp. 13-22). ACM. 
  68. Zhou, R., Khemmarat, S., Gao, L., Wan, J., and Zhang, J. (2016). How YouTube videos are discovered and its impact on video views. Multimedia Tools and Applications, 75(10), 6035-6058.  https://doi.org/10.1007/s11042-015-3206-0
  69. Zhou, T., and Lu, Y. (2011). Examining mobile instant messaging user loyalty from the perspectives of network externalities and flow experience. Computers in Human Behavior, 27(2), 883-889.  https://doi.org/10.1016/j.chb.2010.11.013
  70. Ziegler, C. N., McNee, S. M., Konstan, J. A., and Lausen, G. (2005, May). Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web (pp. 22-32). ACM.