Browse > Article
http://dx.doi.org/10.6109/jkiice.22018.22.4.715

Item Recommendation Technique Using Spark  

Yun, So-Young (Department of Computer Engineering, Pukyong National University)
Youn, Sung-Dae (Department of Computer Engineering, Pukyong National University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.22, no.5, 2018 , pp. 715-721 More about this Journal
Abstract
With the spread of mobile devices, the users of social network services or e-commerce sites have increased dramatically, and the amount of data produced by the users has increased exponentially. E-commerce companies have faced a task regarding how to extract useful information from a vast amount of data produced by the users. To solve this problem, there are various studies applying big data processing technique. In this paper, we propose a collaborative filtering method that applies the tag weight in the Apache Spark platform. In order to elevate the accuracy of recommendation, the proposed method refines the tag data in the preprocessing process and categorizes the items and then applies the information of periods and tag weight to the estimate rating of the items. After generating RDD, we calculate item similarity and prediction values and recommend items to users. The experiment result indicated that the proposed method process large amounts of data quickly and improve the appropriateness of recommendation better.
Keywords
Recommendation Technique; Collaborative Filtering; Apache Spark; Scalability; Tag;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 J.S.Breese, D.Heckerman, and C. Kadle, "Empirical Analysis of Predictive Algorithms for Collaborative Filtering," in Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence, pp.43-52, Jul. 1998.
2 B. Sarwar, G. Karypis, J. Konstan, and J. Riedl, "Item-based Collaborative Filtering Recommendation Algorithms," in Proceedings of the 10th International World Wide Web Conference, ACM Press, pp. 285-295, 2001.
3 O.J. Lee, E.S. You, "Predictive Clustering-based Collaborative Filtering Technique for Performance-Stability of Recommendation System", Journal of Intelligence and Information Systems, vol.21, no. 1 , pp.119-142, Mar. 2015.   DOI
4 S.K. Gorakala, Building Recommendation Engines, Birmingham, Packt Publishing, 2016.
5 S. Yang, B. Wu, "Large Scale Video Data Analysis Based on Spark," 2015 International Conference on Cloud Computing and Big Data(CCBD), pp.209-212, Nov. 2015.
6 M. Zaharia, H. Karau, A. Konwinski, and P. Wendell Learning Spark, Sebastopol. CA: O'REILLY, 2015.
7 U. Ramachandran, H. Venkateswaran, A. Sivasubramaniam, and A. Singla, "Issues in understanding the scalability of parallel systems," in Proceedings of the First International Workshop on Parallel Processing, pp.399-404. Dec. 1994.
8 S.H.H. Ding, B.C.M. Fung and P. Charland, "Kam1n0: MapReduce-based Assembly Clone Search for Reverse Engineering," KDD'16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 461-470, Aug. 2016.
9 A. Stanescu, S. Nagar and D. Caragea "A Hybrid Recommender System: User Profiling from Keywords and Ratings," WI-IAT '13 Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence(WI) and Intelligent Agent Technologies(IAT), vol. 01, pp. 73-80, Nov. 2013.
10 P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl, "Grouplens: an open architecture for collaborative filtering of Netnews," Proceedings of the ACM Conference on Computer Supported Cooperative Work, pp. 175-186, New York, NY, USA, Oct. 1994.
11 G. Linden, B. Smith, and J. York, "Amazon.com recommendations: item-to-item collaborative filtering," IEEE Internet Computing, vol. 7, no. 1, pp. 76-80, January/February 2003.   DOI
12 S. Meng, W. Dou, X. Zhang, and J. Chen, "KASR : A Keyword-Aware Service Recommendation Method on MapReduce for Big Data Applications," IEEE Transactions on Parallel and Distributed Systems, vol. 25, no.12, pp.3221-3231, Dec. 2014.   DOI
13 Y. Shang, Z. Li, W. Qu, Y. Xu, Z. Song, and X. Zhou, "Scalable Collaborative Filtering Recommendation Algorithm with MapReduce," 2014 IEEE 12th International Conference on Dependable, Autonomic and Secure Computing, pp.103-108, Aug. 2014
14 P. Ghuli, A. Ghosh, and R. Shettar, "A collaborative filtering recommendation engine in a distributed environment," 2014 International Conference on Contemporary Computing and Informatics(IC3I), pp. 568-574, Nov. 2014.
15 B.Kupisz, O. Unold, "Collaborative Filtering Recommen- dation Algorithm based on Hadoop and Spark", 2015 IEEE International Conference on Industriial Technology(ICIT), pp.1510-1513, Mar. 2015.
16 C. Sardianos, I. Varlamis, and M. Eirinaki," Scaling Collaborative Filtering to large-scale Bipartite Rating Graphs," 2017 IEEE Third International Conference on Big Data Computing Service and Applications, pp.70-79, Apr. 2017.
17 S. Yun, S. Youn, "Recommendation System Using Big Data Processing Technique," Journal of Korea Institute of Information and Communication Engineering, Vol. 21, No.6, pp.1183-1190 , Jun. 2017.   DOI