• 제목/요약/키워드: sparsity

검색결과 334건 처리시간 0.022초

A Hybrid Recommendation System based on Fuzzy C-Means Clustering and Supervised Learning

  • Duan, Li;Wang, Weiping;Han, Baijing
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제15권7호
    • /
    • pp.2399-2413
    • /
    • 2021
  • A recommendation system is an information filter tool, which uses the ratings and reviews of users to generate a personalized recommendation service for users. However, the cold-start problem of users and items is still a major research hotspot on service recommendations. To address this challenge, this paper proposes a high-efficient hybrid recommendation system based on Fuzzy C-Means (FCM) clustering and supervised learning models. The proposed recommendation method includes two aspects: on the one hand, FCM clustering technique has been applied to the item-based collaborative filtering framework to solve the cold start problem; on the other hand, the content information is integrated into the collaborative filtering. The algorithm constructs the user and item membership degree feature vector, and adopts the data representation form of the scoring matrix to the supervised learning algorithm, as well as by combining the subjective membership degree feature vector and the objective membership degree feature vector in a linear combination, the prediction accuracy is significantly improved on the public datasets with different sparsity. The efficiency of the proposed system is illustrated by conducting several experiments on MovieLens dataset.

오토인코더를 이용한 딥러닝 기반 추천시스템 모형의 비교 연구 (Comparison of deep learning-based autoencoders for recommender systems)

  • 이효진;정윤서
    • 응용통계연구
    • /
    • 제34권3호
    • /
    • pp.329-345
    • /
    • 2021
  • 추천 시스템은 고객의 데이터를 이용하여 개인 맞춤화된 상품을 추천한다. 추천 시스템은 협업 필터링, 콘텐츠 기반 필터링 그리고 이 두 가지를 합친 하이브리드 방법의 세 가지로 크게 나누어진다. 이 연구에서는 딥러닝 방법론에 기초한 오토인코더를 이용한 추천 시스템에 대한 소개와 그 모형들의 비교 연구를 진행한다. 오토인코더는 데이터 행렬에 0이 많은 경우의 문제를 효과적으로 다룰 수 있는 딥러닝 기반의 비지도학습 모형이다. 이 연구에서는 세 개의 실제 데이터를 이용하여 다섯 가지 종류의 오토인코더 기반 모형들을 비교한다. 처음의 세 개 모형은 협업 필터링에 속한 모형이고 나머지 두 개의 모형은 하이브리드 모형이다. 실제 데이터는 고객의 평점 데이터이고, 대부분의 평점이 없어서 희박성 비율이 높다는 특징이 있다.

Effective Pre-rating Method Based on Users' Dichotomous Preferences and Average Ratings Fusion for Recommender Systems

  • Cheng, Shulin;Wang, Wanyan;Yang, Shan;Cheng, Xiufang
    • Journal of Information Processing Systems
    • /
    • 제17권3호
    • /
    • pp.462-472
    • /
    • 2021
  • With an increase in the scale of recommender systems, users' rating data tend to be extremely sparse. Some methods have been utilized to alleviate this problem; nevertheless, it has not been satisfactorily solved yet. Therefore, we propose an effective pre-rating method based on users' dichotomous preferences and average ratings fusion. First, based on a user-item ratings matrix, a new user-item preference matrix was constructed to analyze and model user preferences. The items were then divided into two categories based on a parameterized dynamic threshold. The missing ratings for items that the user was not interested in were directly filled with the lowest user rating; otherwise, fusion ratings were utilized to fill the missing ratings. Further, an optimized parameter λ was introduced to adjust their weights. Finally, we verified our method on a standard dataset. The experimental results show that our method can effectively reduce the prediction error and improve the recommendation quality. As for its application, our method is effective, but not complicated.

Improving Web Service Recommendation using Clustering with K-NN and SVD Algorithms

  • Weerasinghe, Amith M.;Rupasingha, Rupasingha A.H.M.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제15권5호
    • /
    • pp.1708-1727
    • /
    • 2021
  • In the advent of the twenty-first century, human beings began to closely interact with technology. Today, technology is developing, and as a result, the world wide web (www) has a very important place on the Internet and the significant task is fulfilled by Web services. A lot of Web services are available on the Internet and, therefore, it is difficult to find matching Web services among the available Web services. The recommendation systems can help in fixing this problem. In this paper, our observation was based on the recommended method such as the collaborative filtering (CF) technique which faces some failure from the data sparsity and the cold-start problems. To overcome these problems, we first applied an ontology-based clustering and then the k-nearest neighbor (KNN) algorithm for each separate cluster group that effectively increased the data density using the past user interests. Then, user ratings were predicted based on the model-based approach, such as singular value decomposition (SVD) and the predictions used for the recommendation. The evaluation results showed that our proposed approach has a less prediction error rate with high accuracy after analyzing the existing recommendation methods.

Big IoT Healthcare Data Analytics Framework Based on Fog and Cloud Computing

  • Alshammari, Hamoud;El-Ghany, Sameh Abd;Shehab, Abdulaziz
    • Journal of Information Processing Systems
    • /
    • 제16권6호
    • /
    • pp.1238-1249
    • /
    • 2020
  • Throughout the world, aging populations and doctor shortages have helped drive the increasing demand for smart healthcare systems. Recently, these systems have benefited from the evolution of the Internet of Things (IoT), big data, and machine learning. However, these advances result in the generation of large amounts of data, making healthcare data analysis a major issue. These data have a number of complex properties such as high-dimensionality, irregularity, and sparsity, which makes efficient processing difficult to implement. These challenges are met by big data analytics. In this paper, we propose an innovative analytic framework for big healthcare data that are collected either from IoT wearable devices or from archived patient medical images. The proposed method would efficiently address the data heterogeneity problem using middleware between heterogeneous data sources and MapReduce Hadoop clusters. Furthermore, the proposed framework enables the use of both fog computing and cloud platforms to handle the problems faced through online and offline data processing, data storage, and data classification. Additionally, it guarantees robust and secure knowledge of patient medical data.

Sparse decision feedback equalization for underwater acoustic channel based on minimum symbol error rate

  • Wang, Zhenzhong;Chen, Fangjiong;Yu, Hua;Shan, Zhilong
    • International Journal of Naval Architecture and Ocean Engineering
    • /
    • 제13권1호
    • /
    • pp.617-627
    • /
    • 2021
  • Underwater Acoustic Channels (UAC) have inherent sparse characteristics. The traditional adaptive equalization techniques do not utilize this feature to improve the performance. In this paper we consider the Variable Adaptive Subgradient Projection (V-ASPM) method to derive a new sparse equalization algorithm based on the Minimum Symbol Error Rate (MSER) criterion. Compared with the original MSER algorithm, our proposed scheme adds sparse matrix to the iterative formula, which can assign independent step-sizes to the equalizer taps. How to obtain such proper sparse matrix is also analyzed. On this basis, the selection scheme of the sparse matrix is obtained by combining the variable step-sizes and equalizer sparsity measure. We call the new algorithm Sparse-Control Proportional-MSER (SC-PMSER) equalizer. Finally, the proposed SC-PMSER equalizer is embedded into a turbo receiver, which perform turbo decoding, Digital Phase-Locked Loop (DPLL), time-reversal receiving and multi-reception diversity. Simulation and real-field experimental results show that the proposed algorithm has better performance in convergence speed and Bit Error Rate (BER).

Confidence intervals for the COVID-19 neutralizing antibody retention rate in the Korean population

  • Apio, Catherine;Kamruzzaman, Md.;Park, Taesung
    • Genomics & Informatics
    • /
    • 제18권3호
    • /
    • pp.31.1-31.8
    • /
    • 2020
  • The coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has become a global pandemic. No specific therapeutic agents or vaccines for COVID-19 are available, though several antiviral drugs, are under investigation as treatment agents for COVID-19. The use of convalescent plasma transfusion that contain neutralizing antibodies for COVID-19 has become the major focus. This requires mass screening of populations for these antibodies. While several countries started reporting population based antibody rate, its simple point estimate may be misinterpreted without proper estimation of standard error and confidence intervals. In this paper, we review the importance of antibody studies and present the 95% confidence intervals COVID-19 antibody rate for the Korean population using two recently performed antibody tests in Korea. Due to the sparsity of data, the estimation of confidence interval is a big challenge. Thus, we consider several confidence intervals using Asymptotic, Exact and Bayesian estimation methods. In this article, we found that the Wald method gives the narrowest interval among all Asymptotic methods whereas mid p-value gives the narrowest among all Exact methods and Jeffrey's method gives the narrowest from Bayesian method. The most conservative 95% confidence interval estimation shows that as of 00:00 on September 15, 2020, at least 32,602 people were infected but not confirmed in Korea.

Phrase-Chunk Level Hierarchical Attention Networks for Arabic Sentiment Analysis

  • Abdelmawgoud M. Meabed;Sherif Mahdy Abdou;Mervat Hassan Gheith
    • International Journal of Computer Science & Network Security
    • /
    • 제23권9호
    • /
    • pp.120-128
    • /
    • 2023
  • In this work, we have presented ATSA, a hierarchical attention deep learning model for Arabic sentiment analysis. ATSA was proposed by addressing several challenges and limitations that arise when applying the classical models to perform opinion mining in Arabic. Arabic-specific challenges including the morphological complexity and language sparsity were addressed by modeling semantic composition at the Arabic morphological analysis after performing tokenization. ATSA proposed to perform phrase-chunks sentiment embedding to provide a broader set of features that cover syntactic, semantic, and sentiment information. We used phrase structure parser to generate syntactic parse trees that are used as a reference for ATSA. This allowed modeling semantic and sentiment composition following the natural order in which words and phrase-chunks are combined in a sentence. The proposed model was evaluated on three Arabic corpora that correspond to different genres (newswire, online comments, and tweets) and different writing styles (MSA and dialectal Arabic). Experiments showed that each of the proposed contributions in ATSA was able to achieve significant improvement. The combination of all contributions, which makes up for the complete ATSA model, was able to improve the classification accuracy by 3% and 2% on Tweets and Hotel reviews datasets, respectively, compared to the existing models.

유전자 알고리즘을 이용한 클러스터링 기반 협력필터링 (Clustering-based Collaborative Filtering Using Genetic Algorithms)

  • 이수정
    • 창의정보문화연구
    • /
    • 제4권3호
    • /
    • pp.221-230
    • /
    • 2018
  • 추천 시스템의 주요 방법인 협력 필터링 기술은 실제 상업용 온라인 시스템에서 성공적으로 구현되어 서비스가 제공되고 있다. 그러나, 이 기술은 본질적으로 여러 가지 단점을 내포하는데, 데이터 희소성, 콜드 스타트, 확장성 문제 등이 그 예이다. 확장성 문제를 해결하기 위하여 클러스터링 기법을 활용한 협력 필터링 방법이 연구되어 왔다. 본 연구에서 제안하는 협력 필터링 시스템에서는 가장 널리 활용되는 클러스터링 기법들 중 하나인 K-means 알고리즘의 단점을 개선하고자 유전자 알고리즘을 이용한다. 또한, 기존 연구에서 최적화된 클러스터링 결과를 추구하였던 것과는 달리, 제안 방법은 클러스터링 결과를 활용한 협력 필터링 시스템 성능의 최적화를 목표로 하므로, 실질적으로 시스템의 성능을 향상시킬 수 있다.

Compositional data analysis by the square-root transformation: Application to NBA USG% data

  • Jeseok Lee;Byungwon Kim
    • Communications for Statistical Applications and Methods
    • /
    • 제31권3호
    • /
    • pp.349-363
    • /
    • 2024
  • Compositional data refers to data where the sum of the values of the components is a constant, hence the sample space is defined as a simplex making it impossible to apply statistical methods developed in the usual Euclidean vector space. A natural approach to overcome this restriction is to consider an appropriate transformation which moves the sample space onto the Euclidean space, and log-ratio typed transformations, such as the additive log-ratio (ALR), the centered log-ratio (CLR) and the isometric log-ratio (ILR) transformations, have been mostly conducted. However, in scenarios with sparsity, where certain components take on exact zero values, these log-ratio type transformations may not be effective. In this work, we mainly suggest an alternative transformation, that is the square-root transformation which moves the original sample space onto the directional space. We compare the square-root transformation with the log-ratio typed transformation by the simulation study and the real data example. In the real data example, we applied both types of transformations to the USG% data obtained from NBA, and used a density based clustering method, DBSCAN (density-based spatial clustering of applications with noise), to show the result.