• Title/Summary/Keyword: Sparsity

Search Result 335, Processing Time 0.02 seconds

A Hybrid Recommendation System based on Fuzzy C-Means Clustering and Supervised Learning

  • Duan, Li;Wang, Weiping;Han, Baijing
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.7
    • /
    • pp.2399-2413
    • /
    • 2021
  • A recommendation system is an information filter tool, which uses the ratings and reviews of users to generate a personalized recommendation service for users. However, the cold-start problem of users and items is still a major research hotspot on service recommendations. To address this challenge, this paper proposes a high-efficient hybrid recommendation system based on Fuzzy C-Means (FCM) clustering and supervised learning models. The proposed recommendation method includes two aspects: on the one hand, FCM clustering technique has been applied to the item-based collaborative filtering framework to solve the cold start problem; on the other hand, the content information is integrated into the collaborative filtering. The algorithm constructs the user and item membership degree feature vector, and adopts the data representation form of the scoring matrix to the supervised learning algorithm, as well as by combining the subjective membership degree feature vector and the objective membership degree feature vector in a linear combination, the prediction accuracy is significantly improved on the public datasets with different sparsity. The efficiency of the proposed system is illustrated by conducting several experiments on MovieLens dataset.

Comparison of deep learning-based autoencoders for recommender systems (오토인코더를 이용한 딥러닝 기반 추천시스템 모형의 비교 연구)

  • Lee, Hyo Jin;Jung, Yoonsuh
    • The Korean Journal of Applied Statistics
    • /
    • v.34 no.3
    • /
    • pp.329-345
    • /
    • 2021
  • Recommender systems use data from customers to suggest personalized products. The recommender systems can be categorized into three cases; collaborative filtering, contents-based filtering, and hybrid recommender system that combines the first two filtering methods. In this work, we introduce and compare deep learning-based recommender system using autoencoder. Autoencoder is an unsupervised deep learning that can effective solve the problem of sparsity in the data matrix. Five versions of autoencoder-based deep learning models are compared via three real data sets. The first three methods are collaborative filtering and the others are hybrid methods. The data sets are composed of customers' ratings having integer values from one to five. The three data sets are sparse data matrix with many zeroes due to non-responses.

Effective Pre-rating Method Based on Users' Dichotomous Preferences and Average Ratings Fusion for Recommender Systems

  • Cheng, Shulin;Wang, Wanyan;Yang, Shan;Cheng, Xiufang
    • Journal of Information Processing Systems
    • /
    • v.17 no.3
    • /
    • pp.462-472
    • /
    • 2021
  • With an increase in the scale of recommender systems, users' rating data tend to be extremely sparse. Some methods have been utilized to alleviate this problem; nevertheless, it has not been satisfactorily solved yet. Therefore, we propose an effective pre-rating method based on users' dichotomous preferences and average ratings fusion. First, based on a user-item ratings matrix, a new user-item preference matrix was constructed to analyze and model user preferences. The items were then divided into two categories based on a parameterized dynamic threshold. The missing ratings for items that the user was not interested in were directly filled with the lowest user rating; otherwise, fusion ratings were utilized to fill the missing ratings. Further, an optimized parameter λ was introduced to adjust their weights. Finally, we verified our method on a standard dataset. The experimental results show that our method can effectively reduce the prediction error and improve the recommendation quality. As for its application, our method is effective, but not complicated.

Improving Web Service Recommendation using Clustering with K-NN and SVD Algorithms

  • Weerasinghe, Amith M.;Rupasingha, Rupasingha A.H.M.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.5
    • /
    • pp.1708-1727
    • /
    • 2021
  • In the advent of the twenty-first century, human beings began to closely interact with technology. Today, technology is developing, and as a result, the world wide web (www) has a very important place on the Internet and the significant task is fulfilled by Web services. A lot of Web services are available on the Internet and, therefore, it is difficult to find matching Web services among the available Web services. The recommendation systems can help in fixing this problem. In this paper, our observation was based on the recommended method such as the collaborative filtering (CF) technique which faces some failure from the data sparsity and the cold-start problems. To overcome these problems, we first applied an ontology-based clustering and then the k-nearest neighbor (KNN) algorithm for each separate cluster group that effectively increased the data density using the past user interests. Then, user ratings were predicted based on the model-based approach, such as singular value decomposition (SVD) and the predictions used for the recommendation. The evaluation results showed that our proposed approach has a less prediction error rate with high accuracy after analyzing the existing recommendation methods.

Big IoT Healthcare Data Analytics Framework Based on Fog and Cloud Computing

  • Alshammari, Hamoud;El-Ghany, Sameh Abd;Shehab, Abdulaziz
    • Journal of Information Processing Systems
    • /
    • v.16 no.6
    • /
    • pp.1238-1249
    • /
    • 2020
  • Throughout the world, aging populations and doctor shortages have helped drive the increasing demand for smart healthcare systems. Recently, these systems have benefited from the evolution of the Internet of Things (IoT), big data, and machine learning. However, these advances result in the generation of large amounts of data, making healthcare data analysis a major issue. These data have a number of complex properties such as high-dimensionality, irregularity, and sparsity, which makes efficient processing difficult to implement. These challenges are met by big data analytics. In this paper, we propose an innovative analytic framework for big healthcare data that are collected either from IoT wearable devices or from archived patient medical images. The proposed method would efficiently address the data heterogeneity problem using middleware between heterogeneous data sources and MapReduce Hadoop clusters. Furthermore, the proposed framework enables the use of both fog computing and cloud platforms to handle the problems faced through online and offline data processing, data storage, and data classification. Additionally, it guarantees robust and secure knowledge of patient medical data.

Sparse decision feedback equalization for underwater acoustic channel based on minimum symbol error rate

  • Wang, Zhenzhong;Chen, Fangjiong;Yu, Hua;Shan, Zhilong
    • International Journal of Naval Architecture and Ocean Engineering
    • /
    • v.13 no.1
    • /
    • pp.617-627
    • /
    • 2021
  • Underwater Acoustic Channels (UAC) have inherent sparse characteristics. The traditional adaptive equalization techniques do not utilize this feature to improve the performance. In this paper we consider the Variable Adaptive Subgradient Projection (V-ASPM) method to derive a new sparse equalization algorithm based on the Minimum Symbol Error Rate (MSER) criterion. Compared with the original MSER algorithm, our proposed scheme adds sparse matrix to the iterative formula, which can assign independent step-sizes to the equalizer taps. How to obtain such proper sparse matrix is also analyzed. On this basis, the selection scheme of the sparse matrix is obtained by combining the variable step-sizes and equalizer sparsity measure. We call the new algorithm Sparse-Control Proportional-MSER (SC-PMSER) equalizer. Finally, the proposed SC-PMSER equalizer is embedded into a turbo receiver, which perform turbo decoding, Digital Phase-Locked Loop (DPLL), time-reversal receiving and multi-reception diversity. Simulation and real-field experimental results show that the proposed algorithm has better performance in convergence speed and Bit Error Rate (BER).

Confidence intervals for the COVID-19 neutralizing antibody retention rate in the Korean population

  • Apio, Catherine;Kamruzzaman, Md.;Park, Taesung
    • Genomics & Informatics
    • /
    • v.18 no.3
    • /
    • pp.31.1-31.8
    • /
    • 2020
  • The coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has become a global pandemic. No specific therapeutic agents or vaccines for COVID-19 are available, though several antiviral drugs, are under investigation as treatment agents for COVID-19. The use of convalescent plasma transfusion that contain neutralizing antibodies for COVID-19 has become the major focus. This requires mass screening of populations for these antibodies. While several countries started reporting population based antibody rate, its simple point estimate may be misinterpreted without proper estimation of standard error and confidence intervals. In this paper, we review the importance of antibody studies and present the 95% confidence intervals COVID-19 antibody rate for the Korean population using two recently performed antibody tests in Korea. Due to the sparsity of data, the estimation of confidence interval is a big challenge. Thus, we consider several confidence intervals using Asymptotic, Exact and Bayesian estimation methods. In this article, we found that the Wald method gives the narrowest interval among all Asymptotic methods whereas mid p-value gives the narrowest among all Exact methods and Jeffrey's method gives the narrowest from Bayesian method. The most conservative 95% confidence interval estimation shows that as of 00:00 on September 15, 2020, at least 32,602 people were infected but not confirmed in Korea.

Phrase-Chunk Level Hierarchical Attention Networks for Arabic Sentiment Analysis

  • Abdelmawgoud M. Meabed;Sherif Mahdy Abdou;Mervat Hassan Gheith
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.9
    • /
    • pp.120-128
    • /
    • 2023
  • In this work, we have presented ATSA, a hierarchical attention deep learning model for Arabic sentiment analysis. ATSA was proposed by addressing several challenges and limitations that arise when applying the classical models to perform opinion mining in Arabic. Arabic-specific challenges including the morphological complexity and language sparsity were addressed by modeling semantic composition at the Arabic morphological analysis after performing tokenization. ATSA proposed to perform phrase-chunks sentiment embedding to provide a broader set of features that cover syntactic, semantic, and sentiment information. We used phrase structure parser to generate syntactic parse trees that are used as a reference for ATSA. This allowed modeling semantic and sentiment composition following the natural order in which words and phrase-chunks are combined in a sentence. The proposed model was evaluated on three Arabic corpora that correspond to different genres (newswire, online comments, and tweets) and different writing styles (MSA and dialectal Arabic). Experiments showed that each of the proposed contributions in ATSA was able to achieve significant improvement. The combination of all contributions, which makes up for the complete ATSA model, was able to improve the classification accuracy by 3% and 2% on Tweets and Hotel reviews datasets, respectively, compared to the existing models.

Clustering-based Collaborative Filtering Using Genetic Algorithms (유전자 알고리즘을 이용한 클러스터링 기반 협력필터링)

  • Lee, Soojung
    • Journal of Creative Information Culture
    • /
    • v.4 no.3
    • /
    • pp.221-230
    • /
    • 2018
  • Collaborative filtering technique is a major method of recommender systems and has been successfully implemented and serviced in real commercial online systems. However, this technique has several inherent drawbacks, such as data sparsity, cold-start, and scalability problem. Clustering-based collaborative filtering has been studied in order to handle scalability problem. This study suggests a collaborative filtering system which utilizes genetic algorithms to improve shortcomings of K-means algorithm, one of the widely used clustering techniques. Moreover, different from the previous studies that have targeted for optimized clustering results, the proposed method targets the optimization of performance of the collaborative filtering system using the clustering results, which practically can enhance the system performance.

Compositional data analysis by the square-root transformation: Application to NBA USG% data

  • Jeseok Lee;Byungwon Kim
    • Communications for Statistical Applications and Methods
    • /
    • v.31 no.3
    • /
    • pp.349-363
    • /
    • 2024
  • Compositional data refers to data where the sum of the values of the components is a constant, hence the sample space is defined as a simplex making it impossible to apply statistical methods developed in the usual Euclidean vector space. A natural approach to overcome this restriction is to consider an appropriate transformation which moves the sample space onto the Euclidean space, and log-ratio typed transformations, such as the additive log-ratio (ALR), the centered log-ratio (CLR) and the isometric log-ratio (ILR) transformations, have been mostly conducted. However, in scenarios with sparsity, where certain components take on exact zero values, these log-ratio type transformations may not be effective. In this work, we mainly suggest an alternative transformation, that is the square-root transformation which moves the original sample space onto the directional space. We compare the square-root transformation with the log-ratio typed transformation by the simulation study and the real data example. In the real data example, we applied both types of transformations to the USG% data obtained from NBA, and used a density based clustering method, DBSCAN (density-based spatial clustering of applications with noise), to show the result.