• Title/Summary/Keyword: sparsity

Search Result 329, Processing Time 0.025 seconds

A Recommendation Technique using Weight of User Information (사용자 정보 가중치를 이용한 추천 기법)

  • Yun, So-Young;Youn, Sung-Dae
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.15 no.4
    • /
    • pp.877-885
    • /
    • 2011
  • A collaborative filtering(CF) is the most widely used technique in recommender system. However, CF has sparsity and scalability problems. These problems reduce the accuracy of recommendation and extensive studies have been made to solve these problems, In this paper, we proposed a method that uses a weight so as to solve these problems. After creating a user-item matrix, the proposed method analyzes information about users who prefer the item only by using data with a rating over 4 for enhancing the accuracy in the recommendation. The proposed method uses information about the genre of the item as well as analyzed user information as a weight during the calculation of similarity, and it calculates prediction by using only data for which the similarity is over a threshold and uses the data as the rating value of unrated data. It is possible simultaneously to reduce sparsity and to improve accuracy by calculating prediction through an analysis of the characteristics of an item. Also, it is possible to conduct a quick classification based on the analyzed information once a new item and a user are registered. The experiment result indicated that the proposed method has been more enhanced the accuracy, compared to item based, genre based methods.

OLAP System and Performance Evaluation for Analyzing Web Log Data (웹 로그 분석을 위한 OLAP 시스템 및 성능 평가)

  • 김지현;용환승
    • Journal of Korea Multimedia Society
    • /
    • v.6 no.5
    • /
    • pp.909-920
    • /
    • 2003
  • Nowadays, IT for CRM has been growing and developed rapidly. Typical techniques are statistical analysis tools, on-line multidimensional analytical processing (OLAP) tools, and data mining algorithms (such neural networks, decision trees, and association rules). Among customer data, web log data is very important and to use these data efficiently, applying OLAP technology to analyze multi-dimensionally. To make OLAP cube, we have to precalculate multidimensional summary results in order to get fast response. But as the number of dimensions and sparse cells increases, data explosion occurs seriously and the performance of OLAP decreases. In this paper, we presented why the web log data sparsity occurs and then what kinds of sparsity patterns generate in the two and t.he three dimensions for OLAP. Based on this research, we set up the multidimensional data models and query models for benchmark with each sparsity patterns. Finally, we evaluated the performance of three OLAP systems (MS SQL 2000 Analysis Service, Oracle Express and C-MOLAP).

  • PDF

Image Denoising for Metal MRI Exploiting Sparsity and Low Rank Priors

  • Choi, Sangcheon;Park, Jun-Sik;Kim, Hahnsung;Park, Jaeseok
    • Investigative Magnetic Resonance Imaging
    • /
    • v.20 no.4
    • /
    • pp.215-223
    • /
    • 2016
  • Purpose: The management of metal-induced field inhomogeneities is one of the major concerns of distortion-free magnetic resonance images near metallic implants. The recently proposed method called "Slice Encoding for Metal Artifact Correction (SEMAC)" is an effective spin echo pulse sequence of magnetic resonance imaging (MRI) near metallic implants. However, as SEMAC uses the noisy resolved data elements, SEMAC images can have a major problem for improving the signal-to-noise ratio (SNR) without compromising the correction of metal artifacts. To address that issue, this paper presents a novel reconstruction technique for providing an improvement of the SNR in SEMAC images without sacrificing the correction of metal artifacts. Materials and Methods: Low-rank approximation in each coil image is first performed to suppress the noise in the slice direction, because the signal is highly correlated between SEMAC-encoded slices. Secondly, SEMAC images are reconstructed by the best linear unbiased estimator (BLUE), also known as Gauss-Markov or weighted least squares. Noise levels and correlation in the receiver channels are considered for the sake of SNR optimization. To this end, since distorted excitation profiles are sparse, $l_1$ minimization performs well in recovering the sparse distorted excitation profiles and the sparse modeling of our approach offers excellent correction of metal-induced distortions. Results: Three images reconstructed using SEMAC, SEMAC with the conventional two-step noise reduction, and the proposed image denoising for metal MRI exploiting sparsity and low rank approximation algorithm were compared. The proposed algorithm outperformed two methods and produced 119% SNR better than SEMAC and 89% SNR better than SEMAC with the conventional two-step noise reduction. Conclusion: We successfully demonstrated that the proposed, novel algorithm for SEMAC, if compared with conventional de-noising methods, substantially improves SNR and reduces artifacts.

An Agent-based Approach for Distributed Collaborative Filtering (분산 협력 필터링에 대한 에이전트 기반 접근 방법)

  • Kim, Byeong-Man;Li, Qing;Howe Adele E.;Yeo, Dong-Gyu
    • Journal of KIISE:Software and Applications
    • /
    • v.33 no.11
    • /
    • pp.953-964
    • /
    • 2006
  • Due to the usefulness of the collaborative filtering, it has been widely used in both the research and commercial field. However, there are still some challenges for it to be more efficient, especially the scalability problem, the sparsity problem and the cold start problem. In this paper. we address these problems and provide a novel distributed approach based on agents collaboration for the problems. We have tried to solve the scalability problem by making each agent save its users ratings and broadcast them to the users friends so that only friends ratings and his own ratings are kept in an agents local database. To reduce quality degradation of recommendation caused by the lack of rating data, we introduce a method using friends opinions instead of real rating data when they are not available. We also suggest a collaborative filtering algorithm based on user profile to provide new users with recommendation service. Experiments show that our suggested approach is helpful to the new user problem as well as is more scalable than traditional centralized CF filtering systems and alleviate the sparsity problem.

A Music Recommendation System based on Context-awareness using Association Rules (연관규칙을 이용한 상황인식 음악 추천 시스템)

  • Oh, Jae-Taek;Lee, Sang-Yong
    • Journal of Digital Convergence
    • /
    • v.17 no.9
    • /
    • pp.375-381
    • /
    • 2019
  • Recently, the recommendation system has attracted the attention of users as customized recommendation services have been provided focusing on fashion, video and music. But these services are difficult to provide users with proper service according to many different contexts because they do not use contextual information emerging in real time. When applied contextual information expands dimensions, it also increases data sparsity and makes it impossible to recommend proper music for users. Trying to solve these problems, our study proposed a music recommendation system to recommend proper music in real time by applying association rules and using relationships and rules about the current location and time information of users. The accuracy of the recommendation system was measured according to location and time information through 5-fold cross validation. As a result, it was found that the accuracy of the recommendation system was improved as contextual information accumulated.

Applying Different Similarity Measures based on Jaccard Index in Collaborative Filtering

  • Lee, Soojung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.26 no.5
    • /
    • pp.47-53
    • /
    • 2021
  • Sparse ratings data hinder reliable similarity computation between users, which degrades the performance of memory-based collaborative filtering techniques for recommender systems. Many works in the literature have been developed for solving this data sparsity problem, where the most simple and representative ones are the methods of utilizing Jaccard index. This index reflects the number of commonly rated items between two users and is mostly integrated into traditional similarity measures to compute similarity more accurately between the users. However, such integration is very straightforward with no consideration of the degree of data sparsity. This study suggests a novel idea of applying different similarity measures depending on the numeric value of Jaccard index between two users. Performance experiments are conducted to obtain optimal values of the parameters used by the proposed method and evaluate it in comparison with other relevant methods. As a result, the proposed demonstrates the best and comparable performance in prediction and recommendation accuracies.

Similarity Measure based on Utilization of Rating Distributions for Data Sparsity Problem in Collaborative Filtering

  • Lee, Soojung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.12
    • /
    • pp.203-210
    • /
    • 2020
  • Memory-based collaborative filtering is one of the representative types of the recommender system, but it suffers from the inherent problem of data sparsity. Although many works have been devoted to solving this problem, there is still a request for more systematic approaches to the problem. This study exploits distribution of user ratings given to items for computing similarity. All user ratings are utilized in the proposed method, compared to previous ones which use ratings for only common items between users. Moreover, for similarity computation, it takes a global view of ratings for items by reflecting other users' ratings for that item. Performance is evaluated through experiments and compared to that of other relevant methods. The results reveal that the proposed demonstrates superior performance in prediction and rank accuracies. This improvement in prediction accuracy is as high as 2.6 times more than that achieved by the state-of-the-art method over the traditional similarity measures.

Stacked Sparse Autoencoder-DeepCNN Model Trained on CICIDS2017 Dataset for Network Intrusion Detection (네트워크 침입 탐지를 위해 CICIDS2017 데이터셋으로 학습한 Stacked Sparse Autoencoder-DeepCNN 모델)

  • Lee, Jong-Hwa;Kim, Jong-Wouk;Choi, Mi-Jung
    • KNOM Review
    • /
    • v.24 no.2
    • /
    • pp.24-34
    • /
    • 2021
  • Service providers using edge computing provide a high level of service. As a result, devices store important information in inner storage and have become a target of the latest cyberattacks, which are more difficult to detect. Although experts use a security system such as intrusion detection systems, the existing intrusion systems have low detection accuracy. Therefore, in this paper, we proposed a machine learning model for more accurate intrusion detections of devices in edge computing. The proposed model is a hybrid model that combines a stacked sparse autoencoder (SSAE) and a convolutional neural network (CNN) to extract important feature vectors from the input data using sparsity constraints. To find the optimal model, we compared and analyzed the performance as adjusting the sparsity coefficient of SSAE. As a result, the model showed the highest accuracy as a 96.9% using the sparsity constraints. Therefore, the model showed the highest performance when model trains only important features.

Non-homogeneous noise removal for side scan sonar images using a structural sparsity based compressive sensing algorithm (구조적 희소성 기반 압축 센싱 알고리즘을 통한 측면주사소나 영상의 비균일 잡음 제거)

  • Chen, Youngseng;Ku, Bonwha;Lee, Seungho;Kim, Seongil;Ko, Hanseok
    • The Journal of the Acoustical Society of Korea
    • /
    • v.37 no.1
    • /
    • pp.73-81
    • /
    • 2018
  • The quality of side scan sonar images is determined by the frequency of a sonar. A side scan sonar with a low frequency creates low-quality images. One of the factors that lead to low quality is a high-level noise. The noise is occurred by the underwater environment such as equipment noise, signal interference and so on. In addition, in order to compensate for the transmission loss of sonar signals, the received signal is recovered by TVG (Time-Varied Gain), and consequently the side scan sonar images contain non-homogeneous noise which is opposite to optic images whose noise is assumed as homogeneous noise. In this paper, the SSCS (Structural Sparsity based Compressive Sensing) is proposed for removing non-homogeneous noise. The algorithm incorporates both local and non-local models in a structural feature domain so that it guarantees the sparsity and enhances the property of non-local self-similarity. Moreover, the non-local model is corrected in consideration of non-homogeneity of noises. Various experimental results show that the proposed algorithm is superior to existing method.

DOA estimation of signals using non-parametric algorithm (Non-parametric 알고리즘을 이용한 신호의 DOA 추정)

  • 이광식;문성익;양두영
    • Proceedings of the IEEK Conference
    • /
    • 2003.07a
    • /
    • pp.121-124
    • /
    • 2003
  • In this paper, the non-parametric algorithm to estimate DOA(Direction Of Arrival) of signals is proposed and compared with the multidimensional MUSIC algorithm. This non-parametric algorithm with regularizing sparsity constraints achieves super-resolution and noise suppression, effectively. Also, this algorithm offers the increased resolution and significantly reduced sidelobes.

  • PDF