Browse > Article
http://dx.doi.org/10.3745/KIPSTB.2012.19B.1.031

A Reinforcement Learning Approach to Collaborative Filtering Considering Time-sequence of Ratings  

Lee, Jung-Kyu ((주)사이람)
Oh, Byong-Hwa (서강대학교 컴퓨터공학과)
Yang, Ji-Hoon (서강대학교 컴퓨터공학과)
Publication Information
The KIPS Transactions:PartB / v.19B, no.1, 2012 , pp. 31-36 More about this Journal
Abstract
In recent years, there has been increasing interest in recommender systems which provide users with personalized suggestions for products or services. In particular, researches of collaborative filtering analyzing relations between users and items has become more active because of the Netflix Prize competition. This paper presents the reinforcement learning approach for collaborative filtering. By applying reinforcement learning techniques to the movie rating, we discovered the connection between a time sequence of past ratings and current ratings. For this, we first formulated the collaborative filtering problem as a Markov Decision Process. And then we trained the learning model which reflects the connection between the time sequence of past ratings and current ratings using Q-learning. The experimental results indicate that there is a significant effect on current ratings by the time sequence of past ratings.
Keywords
Machine Learning; Recommender Systems; Collaborative Filtering; Reinforcement Learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 R, Salakhutdinov, A. Mnih and G. Hinton, "Restricted Boltzmann machines for collaborative Filtering", Proceedings of the 24th International Conference on Machine Learning, 2007.
2 R. Bell and Y. Koren, "Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights", IEEE International Conference on Data Mining, IEEE, 2007.
3 G. Gorrell and B. Webb, "Generalized hebbian algorithm for incremental latent semantic analysis", Proceedings of Interspeech, 2006.
4 B. Webb, "Netflix update: Try this at home", http://sifter.org/simon/journal/20061211.html, 2006.
5 Y. Koren, "Factorization meets the neighborhood: a multifaceted collaborative filtering model", Proceedings of the 14th ACM SIGKDD international Conference on Knowledge Discovery and Data Mining, pp.426-434, 2008.
6 R. Bellman, "A Markovian Decision Process", Journal of Mathematics and Mechanics 6, 1957.
7 C. Watkins, "Learning from Delayed Rewards", PhD thesis, Cambridge University, Cambridge, England, 1989.
8 MovieLens, http://www.movielens.umn.edu
9 L. Hansen and P. Salamon, "Neural Network Ensembles", IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.12, pp.993-1001, 1990.   DOI   ScienceOn
10 G. Shani, D. Heckerman and R. Brafman, "An MDP-based recommender system", Journal of Machine Learning Research, Vol.6, No.2, pp.1265-1295, 2006.
11 A. Paterek, "Improving regularized singular value decomposition for collaborative Filtering", KDD-Cup and Workshop, ACM press, 2007.
12 B. M. Sarwar and G. Karypis, J. A. Konstan, and J. T Riedl., "Application of Dimensionality Reduction in Recommender System-A Case Study," ACM WebKDD 2000 Web Mining for E-Commerce Workshop, 2000.
13 B. M. Sarwar and G. Karypis, J. A. Konstan, and J.T. Riedl, "Item-based collaborative filtering recommendation algorithms," Proceedings of the 10th international conference on World Wide Web, pp.285-295, 2001.
14 Netflix Prize, http://www.netflixprize.com