Browse > Article
http://dx.doi.org/10.13088/jiis.2019.25.4.089

Evaluating the Quality of Recommendation System by Using Serendipity Measure  

Dorjmaa, Tserendulam (Wonju University Innovation Support Project Team Yonsei University MIRAE Campus)
Shin, Taeksoo (Division of Business Administration College of Government and Business Yonsei University)
Publication Information
Journal of Intelligence and Information Systems / v.25, no.4, 2019 , pp. 89-103 More about this Journal
Abstract
Recently, various approaches to recommendation systems have been studied in terms of the quality of recommendation system. A recommender system basically aims to provide personalized recommendations to users for specific items. Most of these systems always recommend the most relevant items of users or items. Traditionally, the evaluation of recommender system quality has focused on the various predictive accuracy metrics of these. However, recommender system must be not only accurate but also useful to users. User satisfaction with recommender systems as an evaluation criterion of recommender system is related not only to how accurately the system recommends but also to how much it supports the user's decision making. In particular, highly serendipitous recommendation would help a user to find a surprising and interesting item. Serendipity in this study is defined as a measure of the extent to which the recommended items are both attractive and surprising to the users. Therefore, this paper proposes an application of serendipity measure to recommender systems to evaluate the performance of recommender systems in terms of recommendation system quality. In this study we define relevant or attractive unexpectedness as serendipity measure for assessing recommendation systems. That is, serendipity measure is evaluated as the measure indicating how the recommender system can find unexpected and useful items for users. Our experimental results show that highly serendipitous recommendation such as item-based collaborative filtering method has better performance than the other recommendations, i.e. user-based collaborative filtering method in terms of recommendation system quality.
Keywords
Recommendation system; Serendipity measure; Unexpectedness; Relevance; Quality;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Huang, Z., Zeng, D., and Chen, H., "A Link Analysis Approach to Recommendation under Sparse Data," Proceedings of Americas Conference on Information Systems, 2004.
2 Kim, M. S. and Im, I., "Resolving the Gray Sheep Problem Using Social Network Analysis (SNA) in Collaborative Filtering (CF) Recommender Systems," Journal of Intelligent Information System, Vol.20(2014), 137-148.   DOI
3 Linden, G., Smith, B., and York, J., "Amazon.com Recommendation," IEEE Internet Computing, Vol.7, No.1(2003), 76-80.   DOI
4 McNee, S. N., Riedl. J., and Konstan, J. A., "Being Accurate is Not Enough: How Accuracy Metrics have hurt Recommender Systems," Extended Abstracts on Human factors in Computing Systems, CHI(06), (2006), 1097-1101.
5 Murakami, T., Mori, K., and Orihara, R., "Metrics for Evaluating the Serendipity of Recommendation Lists," Proceedings of the Conference on New Frontiers in Artificial Intelligence, (2007), 40-46.
6 Sarwar, B., Karyps, G., Konstan, J., and Reidl, J, "Analysis of Recommendation Algorithms for e-Commerce," Proceedings of the 2nd ACM conference on Electronic Commerce, (2000), 158-167.
7 Sarwar, B., Karyps, G., Konstan, J., and Reidl, J, "Item-based Collaborative Filtering Recommendation Algorithms," Proceedings of the 10th international conference on World Wide Web. ACM, New York, NY, USA, (2001), 285-295.
8 Schafer, J.B., Konstan, J.A., and Riedl, J., "E-Commerce Recommendation Applications," Data Mining and Knowledge Discovery, Vol.5(1/2)(2001), 115-153.   DOI
9 Smith, B. and Linden, G., "Two Decades of Recommender Systems at Amazon.com," IEEE Internet Computing, Vol.21, No.3(2017), 12-18.   DOI
10 Hahsler, M., "recommenderlab: A Framework for Developing and Testing Recommendation Algorithms," Comprehensive R Archive Network, 2014. Available at http://cran.r-project.org/web/packages/recommenderlab/vignettes/recommenderlab.pdf.
11 Herlocker, J.L., Konstan, J.A., Terveen, L.G., and Riedl, J.T, "Evaluating Collaborative Filtering Recommender Systems," ACM Transactions on Information Systems, Vol.22, No.1(2004), 5-53.   DOI
12 Sridharan, S., "Introducing Serendipity in Recommender Systems through Collaborative Methods," Master of Science Thesis, University of Rhode Island, 2014.
13 Su, X. and Khoshgoftaar, T. M., "A Survey of Collaborative Filtering Techniques," Advances in Artificial Intelligence, Vol.2009(2009), 1-19.
14 Chiu, Y. S., Lin, K. H., and Chen, J. S., "A Social Network-based Serendipity Recommender System," International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), (2011).
15 Adamopoulos, P. and Tuzhilin, A., "On Unexpectedness in Recommender Systems: Or How to Better Expect the Unexpected," ACM Transactions on Intelligent Systems and Technology, Vol.1, No.1(2014), 1-51.
16 Adomavicius, G. and Tuzhilin, A., "Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions," IEEE Transactions on Knowledge and Data Engineering, Vol.17, No.6(2005), 734-749.   DOI
17 Aggarwal, C. C., Procopiuc, C., and Yu, P. S., "Finding Localized Associations in Market Basket Data," IEEE Transactions on Knowledge and Data Engineering, Vol.14, No.1(2002), 51-62.   DOI
18 Bernardi, L., Kamps, J., Kiseleva, J., and Muller, M., "The Continuous Cold Start Problem in e-Commerce Recommender Systems," 2nd Workshop on New Trends on Content-Based Recommender Systems, (2015), 30-33.
19 Chen, Y., Wu, C., Xie, M. and Guo, X., "Solving the Sparsity Problem in Recommender Systems Using Association Retrieval," Journal of Computers, Vol.6, No.9(2011), 1896-1902.
20 Deshpande, M. and Karypis, G., "Item-based top-N Recommendation Algorithms," ACM Transactions on Information Systems, Vol.22, No.1(2004), 143-177.   DOI
21 Ge, M., Delgado-Battenfeld, C., and Hannach, D., "Beyond Accuracy: Evaluating Recommender Systems by Coverage and Serendipity," Proceeding of the fourth ACM conference on Recommender systems, (2010), 257-260.
22 Zhang, Y. C., Seaghdha, D. O., Quercia, D., and Jambor, T., "Auralist: Introducing Serendipity into Music Recommendation," Research Note, 2011.
23 Gemmis, M. D., Lops, P., Semeraro, G., and Musto, C., "An Investigation on the Serendipity Problem in Recommendation System," Information Processing and Management, Vol.51(2015), 695-717.   DOI
24 Ghazanfar. M. A. and Prugel-Bennet, A., "Leveraging Clustering Approaches to Solve the Gray-Sheep Users Problem in Recommender Systems," Expert Systems with Applications, Vol.41(2014), 3261-3275.   DOI
25 Surti, T, "Social Recommender Systems: Improving Recommendations through Personalization," Computer Science Department, Haverford College, 2011.