• Title/Summary/Keyword: Sparsity Problem

Search Result 133, Processing Time 0.023 seconds

The Effect of Data Sparsity on Prediction Accuracy in Recommender System (추천시스템의 희소성이 예측 정확도에 미치는 영향에 관한 연구)

  • Kim, Sun-Ok;Lee, Seok-Jun
    • Journal of Internet Computing and Services
    • /
    • v.8 no.6
    • /
    • pp.95-102
    • /
    • 2007
  • Recommender System based on the Collaborative Filtering has a problem of trust of the prediction accuracy because of its problem of sparsity. If the sparsity of a preference value is large, it causes a problem on a process of a choice of neighbors and also lowers the prediction accuracy. In this article, a change of MAE based on the sparsity is studied, groups are classified by sparsity and then, the significant difference among MAEs of classified groups is analyzed. To improve the accuracy of prediction among groups by the problem of sparsity, We studied the improvement of an accurate prediction for recommending system through reducing sparsity by sorting sparsity items, and replacing the average preference among them that has a lot of respondents with the preference evaluation value.

  • PDF

On dual transformation in the interior point method of linear programming (내부점 선형계획법의 쌍대문제 전환에 대하여)

  • 설동렬;박순달;정호원
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 1996.10a
    • /
    • pp.289-292
    • /
    • 1996
  • In Cholesky factorization of the interior point method, dense columns of A matrix make dense Cholesky factor L regardless of sparsity of A matrix. We introduce a method to transform a primal problem to a dual problem in order to preserve the sparsity.

  • PDF

A Simple and Effective Combination of User-Based and Item-Based Recommendation Methods

  • Oh, Se-Chang;Choi, Min
    • Journal of Information Processing Systems
    • /
    • v.15 no.1
    • /
    • pp.127-136
    • /
    • 2019
  • User-based and item-based approaches have been developed as the solutions of the movie recommendation problem. However, the user-based approach is faced with the problem of sparsity, and the item-based approach is faced with the problem of not reflecting users' preferences. In order to solve these problems, there is a research on the combination of the two methods using the concept of similarity. In reality, it is not free from the problem of sparsity, since it has a lot of parameters to be calculated. In this study, we propose a combining method that simplifies the combination equation of prior study. This method is relatively free from the problem of sparsity, since it has less parameters to be calculated. Thus, it can get more accurate results by reflecting the users rating to calculate the parameters. It is very fast to predict new movie ratings as well. In experiments for the proposed method, the initial error is large, but the performance gets quickly stabilized after. In addition, it showed about 6% lower average error rate than the existing method using similarity.

Chaotic Features for Dynamic Textures Recognition with Group Sparsity Representation

  • Luo, Xinbin;Fu, Shan;Wang, Yong
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.9 no.11
    • /
    • pp.4556-4572
    • /
    • 2015
  • Dynamic texture (DT) recognition is a challenging problem in numerous applications. In this study, we propose a new algorithm for DT recognition based on group sparsity structure in conjunction with chaotic feature vector. Bag-of-words model is used to represent each video as a histogram of the chaotic feature vector, which is proposed to capture self-similarity property of the pixel intensity series. The recognition problem is then cast to a group sparsity model, which can be efficiently optimized through alternating direction method of multiplier algorithm. Experimental results show that the proposed method exhibited the best performance among several well-known DT modeling techniques.

Power Failure Sensitivity Analysis via Grouped L1/2 Sparsity Constrained Logistic Regression

  • Li, Baoshu;Zhou, Xin;Dong, Ping
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.8
    • /
    • pp.3086-3101
    • /
    • 2021
  • To supply precise marketing and differentiated service for the electric power service department, it is very important to predict the customers with high sensitivity of electric power failure. To solve this problem, we propose a novel grouped 𝑙1/2 sparsity constrained logistic regression method for sensitivity assessment of electric power failure. Different from the 𝑙1 norm and k-support norm, the proposed grouped 𝑙1/2 sparsity constrained logistic regression method simultaneously imposes the inter-class information and tighter approximation to the nonconvex 𝑙0 sparsity to exploit multiple correlated attributions for prediction. Firstly, the attributes or factors for predicting the customer sensitivity of power failure are selected from customer sheets, such as customer information, electric consuming information, electrical bill, 95598 work sheet, power failure events, etc. Secondly, all these samples with attributes are clustered into several categories, and samples in the same category are assumed to be sharing similar properties. Then, 𝑙1/2 norm constrained logistic regression model is built to predict the customer's sensitivity of power failure. Alternating direction of multipliers (ADMM) algorithm is finally employed to solve the problem by splitting it into several sub-problems effectively. Experimental results on power electrical dataset with about one million customer data from a province validate that the proposed method has a good prediction accuracy.

A Movie Recommendation Method Using Rating Difference Between Items (항목 간 선호도 차이를 이용한 영화 추천 방법)

  • Oh, Se-Chang;Choi, Min
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.17 no.11
    • /
    • pp.2602-2608
    • /
    • 2013
  • User-based and item-based method have been developed as the solutions of the movie recommendation problem. However, these methods are faced with the sparsity problem and the problem of not reflecting user's rating respectively. In order to solve these problems, there is a research on the combination of the two methods using the concept of similarity. In reality, it is not free from the problem of sparsity, since it has a lot of parameters to be calculated. In this study, we propose a recommendation method using rating difference between items in order to complement this problem. This method is relatively free from the problem of sparsity, since it has less parameters to be calculated. And it can get more accurate results by reflecting the users rating to calculate the parameters. In experiments for the proposed method, the initial error is large, but the performance has been quickly stabilized after. In addition, it showed a 0.0538 lower average error compared to the existing method using similarity.

Applying Centrality Analysis to Solve the Cold-Start and Sparsity Problems in Collaborative Filtering (협업필터링의 신규고객추천 및 희박성 문제 해결을 위한 중심성분석의 활용)

  • Cho, Yoon-Ho;Bang, Joung-Hae
    • Journal of Intelligence and Information Systems
    • /
    • v.17 no.3
    • /
    • pp.99-114
    • /
    • 2011
  • Collaborative Filtering (CF) suffers from two major problems:sparsity and cold-start recommendation. This paper focuses on the cold-start problem for new customers with no purchase records and the sparsity problem for the customers with very few purchase records. For the purpose, we propose a method for the new customer recommendation by using a combined measure based on three well-used centrality measures to identify the customers who are most likely to become neighbors of the new customer. To alleviate the sparsity problem, we also propose a hybrid approach that applies our method to customers with very few purchase records and CF to the other customers with sufficient purchases. To evaluate the effectiveness of our method, we have conducted several experiments using a data set from a department store in Korea. The experiment results show that the combination of two measures makes better recommendations than not only a single measure but also the best-seller-based method and that the performance is improved when applying the hybrid approach.

Method to Improve Data Sparsity Problem of Collaborative Filtering Using Latent Attribute Preference (잠재적 속성 선호도를 이용한 협업 필터링의 데이터 희소성 문제 개선 방법)

  • Kwon, Hyeong-Joon;Hong, Kwang-Seok
    • Journal of Internet Computing and Services
    • /
    • v.14 no.5
    • /
    • pp.59-67
    • /
    • 2013
  • In this paper, we propose the LAR_CF, latent attribute rating-based collaborative filtering, that is robust to data sparsity problem which is one of traditional problems caused of decreasing rating prediction accuracy. As compared with that existing collaborative filtering method uses a preference rating rated by users as feature vector to calculate similarity between objects, the proposed method improves data sparsity problem using unique attributes of two target objects with existing explicit preference. We consider MovieLens 100k dataset and its item attributes to evaluate the LAR_CF. As a result of artificial data sparsity and full-rating experiments, we confirmed that rating prediction accuracy can be improved rating prediction accuracy in data sparsity condition by the LAR_CF.

Sparsity Adaptive Expectation Maximization Algorithm for Estimating Channels in MIMO Cooperation systems

  • Zhang, Aihua;Yang, Shouyi;Li, Jianjun;Li, Chunlei;Liu, Zhoufeng
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.8
    • /
    • pp.3498-3511
    • /
    • 2016
  • We investigate the channel state information (CSI) in multi-input multi-output (MIMO) cooperative networks that employ the amplify-and-forward transmission scheme. Least squares and expectation conditional maximization have been proposed in the system. However, neither of these two approaches takes advantage of channel sparsity, and they cause estimation performance loss. Unlike linear channel estimation methods, several compressed channel estimation methods are proposed in this study to exploit the sparsity of the MIMO cooperative channels based on the theory of compressed sensing. First, the channel estimation problem is formulated as a compressed sensing problem by using sparse decomposition theory. Second, the lower bound is derived for the estimation, and the MIMO relay channel is reconstructed via compressive sampling matching pursuit algorithms. Finally, based on this model, we propose a novel algorithm so called sparsity adaptive expectation maximization (SAEM) by using Kalman filter and expectation maximization algorithm so that it can exploit channel sparsity alternatively and also track the true support set of time-varying channel. Kalman filter is used to provide soft information of transmitted signals to the EM-based algorithm. Various numerical simulation results indicate that the proposed sparse channel estimation technique outperforms the previous estimation schemes.

Multiview-based Spectral Weighted and Low-Rank for Row-sparsity Hyperspectral Unmixing

  • Zhang, Shuaiyang;Hua, Wenshen;Liu, Jie;Li, Gang;Wang, Qianghui
    • Current Optics and Photonics
    • /
    • v.5 no.4
    • /
    • pp.431-443
    • /
    • 2021
  • Sparse unmixing has been proven to be an effective method for hyperspectral unmixing. Hyperspectral images contain rich spectral and spatial information. The means to make full use of spectral information, spatial information, and enhanced sparsity constraints are the main research directions to improve the accuracy of sparse unmixing. However, many algorithms only focus on one or two of these factors, because it is difficult to construct an unmixing model that considers all three factors. To address this issue, a novel algorithm called multiview-based spectral weighted and low-rank row-sparsity unmixing is proposed. A multiview data set is generated through spectral partitioning, and then spectral weighting is imposed on it to exploit the abundant spectral information. The row-sparsity approach, which controls the sparsity by the l2,0 norm, outperforms the single-sparsity approach in many scenarios. Many algorithms use convex relaxation methods to solve the l2,0 norm to avoid the NP-hard problem, but this will reduce sparsity and unmixing accuracy. In this paper, a row-hard-threshold function is introduced to solve the l2,0 norm directly, which guarantees the sparsity of the results. The high spatial correlation of hyperspectral images is associated with low column rank; therefore, the low-rank constraint is adopted to utilize spatial information. Experiments with simulated and real data prove that the proposed algorithm can obtain better unmixing results.