• Title/Summary/Keyword: Collaborative Filtering algorithms

Search Result 60, Processing Time 0.026 seconds

Pre-Evaluation for Prediction Accuracy by Using the Customer's Ratings in Collaborative Filtering (협업필터링에서 고객의 평가치를 이용한 선호도 예측의 사전평가에 관한 연구)

  • Lee, Seok-Jun;Kim, Sun-Ok
    • Asia pacific journal of information systems
    • /
    • v.17 no.4
    • /
    • pp.187-206
    • /
    • 2007
  • The development of computer and information technology has been combined with the information superhighway internet infrastructure, so information widely spreads not only in special fields but also in the daily lives of people. Information ubiquity influences the traditional way of transaction, and leads a new E-commerce which distinguishes from the existing E-commerce. Not only goods as physical but also service as non-physical come into E-commerce. As the scale of E-Commerce is being enlarged as well. It keeps people from finding information they want. Recommender systems are now becoming the main tools for E-Commerce to mitigate the information overload. Recommender systems can be defined as systems for suggesting some Items(goods or service) considering customers' interests or tastes. They are being used by E-commerce web sites to suggest products to their customers who want to find something for them and to provide them with information to help them decide which to purchase. There are several approaches of recommending goods to customer in recommender system but in this study, the main subject is focused on collaborative filtering technique. This study presents a possibility of pre-evaluation for the prediction performance of customer's preference in collaborative filtering before the process of customer's preference prediction. Pre-evaluation for the prediction performance of each customer having low performance is classified by using the statistical features of ratings rated by each customer is conducted before the prediction process. In this study, MovieLens 100K dataset is used to analyze the accuracy of classification. The classification criteria are set by using the training sets divided 80% from the 100K dataset. In the process of classification, the customers are divided into two groups, classified group and non classified group. To compare the prediction performance of classified group and non classified group, the prediction process runs the 20% test set through the Neighborhood Based Collaborative Filtering Algorithm and Correspondence Mean Algorithm. The prediction errors from those prediction algorithm are allocated to each customer and compared with each user's error. Research hypothesis : Two research hypotheses are formulated in this study to test the accuracy of the classification criterion as follows. Hypothesis 1: The estimation accuracy of groups classified according to the standard deviation of each user's ratings has significant difference. To test the Hypothesis 1, the standard deviation is calculated for each user in training set which is divided 80% from MovieLens 100K dataset. Four groups are classified according to the quartile of the each user's standard deviations. It is compared to test the estimation errors of each group which results from test set are significantly different. Hypothesis 2: The estimation accuracy of groups that are classified according to the distribution of each user's ratings have significant differences. To test the Hypothesis 2, the distributions of each user's ratings are compared with the distribution of ratings of all customers in training set which is divided 80% from MovieLens 100K dataset. It assumes that the customers whose ratings' distribution are different from that of all customers would have low performance, so six types of different distributions are set to be compared. The test groups are classified into fit group or non-fit group according to the each type of different distribution assumed. The degrees in accordance with each type of distribution and each customer's distributions are tested by the test of ${\chi}^2$ goodness-of-fit and classified two groups for testing the difference of the mean of errors. Also, the degree of goodness-of-fit with the distribution of each user's ratings and the average distribution of the ratings in the training set are closely related to the prediction errors from those prediction algorithms. Through this study, the customers who have lower performance of prediction than the rest in the system are classified by those two criteria, which are set by statistical features of customers ratings in the training set, before the prediction process.

Development of Web-based Intelligent Recommender Systems using Advanced Data Mining Techniques (개선된 데이터 마이닝 기술에 의한 웹 기반 지능형 추천시스템 구축)

  • Kim Kyoung-Jae;Ahn Hyunchul
    • Journal of Information Technology Applications and Management
    • /
    • v.12 no.3
    • /
    • pp.41-56
    • /
    • 2005
  • Product recommender system is one of the most popular techniques for customer relationship management. In addition, collaborative filtering (CF) has been known to be one of the most successful recommendation techniques in product recommender systems. However, CF has some limitations such as sparsity and scalability problems. This study proposes hybrid cluster analysis and case-based reasoning (CBR) to address these problems. CBR may relieve the sparsity problem because it recommends products using customer profile and transaction data, but it may still give rise to scalability problem. Thus, this study uses cluster analysis to reduce search space prior to CBR for scalability Problem. For cluster analysis, this study employs hybrid genetic and K-Means algorithms to avoid possibility of convergence in local minima of typical cluster analyses. This study also develops a Web-based prototype system to test the superiority of the proposed model.

  • PDF

A Comparative Study on Over-The-Tops, Netflix & Amazon Prime Video: Based on the Success Factors of Innovation

  • Song, Minzheong
    • International journal of advanced smart convergence
    • /
    • v.10 no.1
    • /
    • pp.62-74
    • /
    • 2021
  • We compare Over-the-Tops (OTTs), Netflix and Amazon Prime Video (APV) with five success factors of innovation. Firstly, Netflix offers better personalized service than APV, because APV has collaborative filtering algorithms to recommend safe bets, not the customers really want. Secondly, APV' user interface is undercooked to lock the members in, even if it has more content and better price offer than Netflix retaining its loyal customers despite the price increase. Thirdly, Netflix has simple subscription model with three tiering, but APV has complicated pricing model having annual and monthly, APV and Prime Video (AV) app, Amazon subscription and extra payment of Amazon Prime Channels (APCs). Fourthly, Amazon has fewer partnership than Netflix especially when it comes to local TV series. Instead, Amazon has live TV channel collaboration including sports content. Lastly, both have strategic and operational agility in their organization well.

Recommending Talks at International Research Conferences (국제학술대회 참가자들을 위한 정보추천 서비스)

  • Lee, Danielle H.
    • Journal of Intelligence and Information Systems
    • /
    • v.18 no.3
    • /
    • pp.13-34
    • /
    • 2012
  • The Paper Explores The Problem Of Recommending Talks To Attend At International Research Conferences. When Researchers Participate In Conferences, Finding Interesting Talks To Attend Is A Real Challenge. Given That Several Presentation Sessions And Social Activities Are Typically Held At A Time, And There Is Little Time To Analyze All Alternatives, It Is Easy To Miss Important Talks. In Addition, Compared With Recommendations Of Products Such As Movies, Books, Music, Etc. The Recipients Of Talk Recommendations (i.e. Conference Attendees) Already Formed Their Own Research Community On The Center Of The Conference Topics. Hence, Recommending Conference Talks Contains Highly Social Context. This Study Suggests That This Domain Would Be Suitable For Social Network-Based Recommendations. In Order To Find Out The Most Effective Recommendation Approach, Three Sources Of Information Were Explored For Talk Recommendation-Whateach Talk Is About (Content), Who Scheduled The Talks (Collaborative), And How The Users Are Connected Socially (Social). Using These Three Sources Of Information, This Paper Examined Several Direct And Hybrid Recommendation Algorithms To Help Users Find Interesting Talks More Easily. Using A Dataset Of A Conference Scheduling System, Conference Navigator, Multiple Approaches Ranging From Classic Content-Based And Collaborative Filtering Recommendations To Social Network-Based Recommendations Were Compared. As The Result, For Cold-Start Users Who Have Insufficient Number Of Items To Express Their Preferences, The Recommendations Based On Their Social Networks Generated The Best Suggestions.

Predicting the Performance of Recommender Systems through Social Network Analysis and Artificial Neural Network (사회연결망분석과 인공신경망을 이용한 추천시스템 성능 예측)

  • Cho, Yoon-Ho;Kim, In-Hwan
    • Journal of Intelligence and Information Systems
    • /
    • v.16 no.4
    • /
    • pp.159-172
    • /
    • 2010
  • The recommender system is one of the possible solutions to assist customers in finding the items they would like to purchase. To date, a variety of recommendation techniques have been developed. One of the most successful recommendation techniques is Collaborative Filtering (CF) that has been used in a number of different applications such as recommending Web pages, movies, music, articles and products. CF identifies customers whose tastes are similar to those of a given customer, and recommends items those customers have liked in the past. Numerous CF algorithms have been developed to increase the performance of recommender systems. Broadly, there are memory-based CF algorithms, model-based CF algorithms, and hybrid CF algorithms which combine CF with content-based techniques or other recommender systems. While many researchers have focused their efforts in improving CF performance, the theoretical justification of CF algorithms is lacking. That is, we do not know many things about how CF is done. Furthermore, the relative performances of CF algorithms are known to be domain and data dependent. It is very time-consuming and expensive to implement and launce a CF recommender system, and also the system unsuited for the given domain provides customers with poor quality recommendations that make them easily annoyed. Therefore, predicting the performances of CF algorithms in advance is practically important and needed. In this study, we propose an efficient approach to predict the performance of CF. Social Network Analysis (SNA) and Artificial Neural Network (ANN) are applied to develop our prediction model. CF can be modeled as a social network in which customers are nodes and purchase relationships between customers are links. SNA facilitates an exploration of the topological properties of the network structure that are implicit in data for CF recommendations. An ANN model is developed through an analysis of network topology, such as network density, inclusiveness, clustering coefficient, network centralization, and Krackhardt's efficiency. While network density, expressed as a proportion of the maximum possible number of links, captures the density of the whole network, the clustering coefficient captures the degree to which the overall network contains localized pockets of dense connectivity. Inclusiveness refers to the number of nodes which are included within the various connected parts of the social network. Centralization reflects the extent to which connections are concentrated in a small number of nodes rather than distributed equally among all nodes. Krackhardt's efficiency characterizes how dense the social network is beyond that barely needed to keep the social group even indirectly connected to one another. We use these social network measures as input variables of the ANN model. As an output variable, we use the recommendation accuracy measured by F1-measure. In order to evaluate the effectiveness of the ANN model, sales transaction data from H department store, one of the well-known department stores in Korea, was used. Total 396 experimental samples were gathered, and we used 40%, 40%, and 20% of them, for training, test, and validation, respectively. The 5-fold cross validation was also conducted to enhance the reliability of our experiments. The input variable measuring process consists of following three steps; analysis of customer similarities, construction of a social network, and analysis of social network patterns. We used Net Miner 3 and UCINET 6.0 for SNA, and Clementine 11.1 for ANN modeling. The experiments reported that the ANN model has 92.61% estimated accuracy and 0.0049 RMSE. Thus, we can know that our prediction model helps decide whether CF is useful for a given application with certain data characteristics.

Enhancing Predictive Accuracy of Collaborative Filtering Algorithms using the Network Analysis of Trust Relationship among Users (사용자 간 신뢰관계 네트워크 분석을 활용한 협업 필터링 알고리즘의 예측 정확도 개선)

  • Choi, Seulbi;Kwahk, Kee-Young;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.3
    • /
    • pp.113-127
    • /
    • 2016
  • Among the techniques for recommendation, collaborative filtering (CF) is commonly recognized to be the most effective for implementing recommender systems. Until now, CF has been popularly studied and adopted in both academic and real-world applications. The basic idea of CF is to create recommendation results by finding correlations between users of a recommendation system. CF system compares users based on how similar they are, and recommend products to users by using other like-minded people's results of evaluation for each product. Thus, it is very important to compute evaluation similarities among users in CF because the recommendation quality depends on it. Typical CF uses user's explicit numeric ratings of items (i.e. quantitative information) when computing the similarities among users in CF. In other words, user's numeric ratings have been a sole source of user preference information in traditional CF. However, user ratings are unable to fully reflect user's actual preferences from time to time. According to several studies, users may more actively accommodate recommendation of reliable others when purchasing goods. Thus, trust relationship can be regarded as the informative source for identifying user's preference with accuracy. Under this background, we propose a new hybrid recommender system that fuses CF and social network analysis (SNA). The proposed system adopts the recommendation algorithm that additionally reflect the result analyzed by SNA. In detail, our proposed system is based on conventional memory-based CF, but it is designed to use both user's numeric ratings and trust relationship information between users when calculating user similarities. For this, our system creates and uses not only user-item rating matrix, but also user-to-user trust network. As the methods for calculating user similarity between users, we proposed two alternatives - one is algorithm calculating the degree of similarity between users by utilizing in-degree and out-degree centrality, which are the indices representing the central location in the social network. We named these approaches as 'Trust CF - All' and 'Trust CF - Conditional'. The other alternative is the algorithm reflecting a neighbor's score higher when a target user trusts the neighbor directly or indirectly. The direct or indirect trust relationship can be identified by searching trust network of users. In this study, we call this approach 'Trust CF - Search'. To validate the applicability of the proposed system, we used experimental data provided by LibRec that crawled from the entire FilmTrust website. It consists of ratings of movies and trust relationship network indicating who to trust between users. The experimental system was implemented using Microsoft Visual Basic for Applications (VBA) and UCINET 6. To examine the effectiveness of the proposed system, we compared the performance of our proposed method with one of conventional CF system. The performances of recommender system were evaluated by using average MAE (mean absolute error). The analysis results confirmed that in case of applying without conditions the in-degree centrality index of trusted network of users(i.e. Trust CF - All), the accuracy (MAE = 0.565134) was lower than conventional CF (MAE = 0.564966). And, in case of applying the in-degree centrality index only to the users with the out-degree centrality above a certain threshold value(i.e. Trust CF - Conditional), the proposed system improved the accuracy a little (MAE = 0.564909) compared to traditional CF. However, the algorithm searching based on the trusted network of users (i.e. Trust CF - Search) was found to show the best performance (MAE = 0.564846). And the result from paired samples t-test presented that Trust CF - Search outperformed conventional CF with 10% statistical significance level. Our study sheds a light on the application of user's trust relationship network information for facilitating electronic commerce by recommending proper items to users.

Recognition Method of Korean Abnormal Language for Spam Mail Filtering (스팸메일 필터링을 위한 한글 변칙어 인식 방법)

  • Ahn, Hee-Kook;Han, Uk-Pyo;Shin, Seung-Ho;Yang, Dong-Il;Roh, Hee-Young
    • Journal of Advanced Navigation Technology
    • /
    • v.15 no.2
    • /
    • pp.287-297
    • /
    • 2011
  • As electronic mails are being widely used for facility and speedness of information communication, as the amount of spam mails which have malice and advertisement increase and cause lots of social and economic problem. A number of approaches have been proposed to alleviate the impact of spam. These approaches can be categorized into pre-acceptance and post-acceptance methods. Post-acceptance methods include bayesian filters, collaborative filtering and e-mail prioritization which are based on words or sentances. But, spammers are changing those characteristics and sending to avoid filtering system. In the case of Korean, the abnormal usages can be much more than other languages because syllable is composed of chosung, jungsung, and jongsung. Existing formal expressions and learning algorithms have the limits to meet with those changes promptly and efficiently. So, we present an methods for recognizing Korean abnormal language(Koral) to improve accuracy and efficiency of filtering system. The method is based on syllabic than word and Smith-waterman algorithm. Through the experiment on filter keyword and e-mail extracted from mail server, we confirmed that Koral is recognized exactly according to similarity level. The required time and space costs are within the permitted limit.

A review on robust principal component analysis (강건 주성분분석에 대한 요약)

  • Lee, Eunju;Park, Mingyu;Kim, Choongrak
    • The Korean Journal of Applied Statistics
    • /
    • v.35 no.2
    • /
    • pp.327-333
    • /
    • 2022
  • Principal component analysis (PCA) is the most widely used technique in dimension reduction, however, it is very sensitive to outliers. A robust version of PCA, called robust PCA, was suggested by two seminal papers by Candès et al. (2011) and Chandrasekaran et al. (2011). The robust PCA is an essential tool in the artificial intelligence such as background detection, face recognition, ranking, and collaborative filtering. Also, the robust PCA receives a lot of attention in statistics in addition to computer science. In this paper, we introduce recent algorithms for the robust PCA and give some illustrative examples.

Analysis of Preference Criteria for Personalized Web Search (개인화된 웹 검색을 위한 선호 기준 분석)

  • Lee, Soo-Jung
    • The Journal of Korean Association of Computer Education
    • /
    • v.13 no.1
    • /
    • pp.45-52
    • /
    • 2010
  • With rapid increase in the number of web documents, the problem of information overload in Internet search is growing seriously. In order to improve web search results, previous research studies employed user queries/preferred words and the number of links in the web documents. In this study, performance of the search results exploiting these two criteria is examined and other preference criteria for web documents are analyzed. Experimental results show that personalized web search results employing queries and preferred words yield up to 1.7 times better performance over the current search engine and that the search results using the number of links gives up to 1.3 times better performance. Although it is found that the first of the user's preference criteria for web documents is the contents of the document, readability and images in the document are also given a large weight. Therefore, performance of web search personalization algorithms will be greatly improved if they incorporate objective data reflecting each user's characteristics in addition to the number of queries and preferred words.

  • PDF

Comparison of online video(OTT) content production technology based on artificial intelligence customized recommendation service (인공지능 맞춤 추천서비스 기반 온라인 동영상(OTT) 콘텐츠 제작 기술 비교)

  • CHUN, Sanghun;SHIN, Seoung-Jung
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.21 no.3
    • /
    • pp.99-105
    • /
    • 2021
  • In addition to the OTT video production service represented by Nexflix and YouTube, a personalized recommendation system for content with artificial intelligence has become common. YouTube's personalized recommendation service system consists of two neural networks, one neural network consisting of a recommendation candidate generation model and the other consisting of a ranking network. Netflix's video recommendation system consists of two data classification systems, divided into content-based filtering and collaborative filtering. As the online platform-led content production is activated by the Corona Pandemic, the field of virtual influencers using artificial intelligence is emerging. Virtual influencers are produced with GAN (Generative Adversarial Networks) artificial intelligence, and are unsupervised learning algorithms in which two opposing systems compete with each other. This study also researched the possibility of developing AI platform based on individual recommendation and virtual influencer (metabus) as a core content of OTT in the future.