• Title/Summary/Keyword: clustering by unsupervised learning

Search Result 86, Processing Time 0.024 seconds

Classification of Traffic Flows into QoS Classes by Unsupervised Learning and KNN Clustering

  • Zeng, Yi;Chen, Thomas M.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.3 no.2
    • /
    • pp.134-146
    • /
    • 2009
  • Traffic classification seeks to assign packet flows to an appropriate quality of service(QoS) class based on flow statistics without the need to examine packet payloads. Classification proceeds in two steps. Classification rules are first built by analyzing traffic traces, and then the classification rules are evaluated using test data. In this paper, we use self-organizing map and K-means clustering as unsupervised machine learning methods to identify the inherent classes in traffic traces. Three clusters were discovered, corresponding to transactional, bulk data transfer, and interactive applications. The K-nearest neighbor classifier was found to be highly accurate for the traffic data and significantly better compared to a minimum mean distance classifier.

A Study on Classification Evaluation Prediction Model by Cluster for Accuracy Measurement of Unsupervised Learning Data (비지도학습 데이터의 정확성 측정을 위한 클러스터별 분류 평가 예측 모델에 대한 연구)

  • Jung, Se Hoon;Kim, Jong Chan;Kim, Cheeyong;You, Kang Soo;Sim, Chun Bo
    • Journal of Korea Multimedia Society
    • /
    • v.21 no.7
    • /
    • pp.779-786
    • /
    • 2018
  • In this paper, we are applied a nerve network to allow for the reflection of data learning methods in their overall forms by using cluster data rather than data learning by the stages and then selected a nerve network model and analyzed its variables through learning by the cluster. The CkLR algorithm was proposed to analyze the reaction variables of clustering outcomes through an approach to the initialization of K-means clustering and build a model to assess the prediction rate of clustering and the accuracy rate of prediction in case of new data inputs. The performance evaluation results show that the accuracy rate of test data by the class was over 92%, which was the mean accuracy rate of the entire test data, thus confirming the advantages of a specialized structure found in the proposed learning nerve network by the class.

Analysis on the Distribution of RF Threats Using Unsupervised Learning Techniques (비지도 학습 기법을 사용한 RF 위협의 분포 분석)

  • Kim, Chulpyo;Noh, Sanguk;Park, So Ryoung
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.19 no.3
    • /
    • pp.346-355
    • /
    • 2016
  • In this paper, we propose a method to analyze the clusters of RF threats emitting electrical signals based on collected signal variables in integrated electronic warfare environments. We first analyze the signal variables collected by an electronic warfare receiver, and construct a model based on variables showing the properties of threats. To visualize the distribution of RF threats and reversely identify them, we use k-means clustering algorithm and self-organizing map (SOM) algorithm, which are belonging to unsupervised learning techniques. Through the resulting model compiled by k-means clustering and SOM algorithms, the RF threats can be classified into one of the distribution of RF threats. In an experiment, we measure the accuracy of classification results using the algorithms, and verify the resulting model that could be used to visually recognize the distribution of RF threats.

Development of an unsupervised learning-based ESG evaluation process for Korean public institutions without label annotation

  • Do Hyeok Yoo;SuJin Bak
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.5
    • /
    • pp.155-164
    • /
    • 2024
  • This study proposes an unsupervised learning-based clustering model to estimate the ESG ratings of domestic public institutions. To achieve this, the optimal number of clusters was determined by comparing spectral clustering and k-means clustering. These results are guaranteed by calculating the Davies-Bouldin Index (DBI), a model performance index. The DBI values were 0.734 for spectral clustering and 1.715 for k-means clustering, indicating lower values showed better performance. Thus, the superiority of spectral clustering was confirmed. Furthermore, T-test and ANOVA were used to reveal statistically significant differences between ESG non-financial data, and correlation coefficients were used to confirm the relationships between ESG indicators. Based on these results, this study suggests the possibility of estimating the ESG performance ranking of each public institution without existing ESG ratings. This is achieved by calculating the optimal number of clusters, and then determining the sum of averages of the ESG data within each cluster. Therefore, the proposed model can be employed to evaluate the ESG ratings of various domestic public institutions, and it is expected to be useful in domestic sustainable management practice and performance management.

Understanding postal delivery areas in the Republic of Korea using multiple unsupervised learning approaches

  • Han, Keejun;Yu, Yeongwoong;Na, Dong-gil;Jung, Hoon;Heo, Younggyo;Jeong, Hyeoncheol;Yun, Sunguk;Kim, Jungeun
    • ETRI Journal
    • /
    • v.44 no.2
    • /
    • pp.232-243
    • /
    • 2022
  • Changes in household composition and the residential environment have had a considerable impact on the features of postal delivery regions in recent years, resulting in a large increase in the overall workload of domestic postal delivery services. In this paper, we provide complex analysis results for postal delivery areas using various unsupervised learning approaches. First, we extract highly influential features using several feature-engineering methods. Then, using quantitative and qualitative cluster analyses, we find the distinctive traits and semantics of postal delivery zones. Unsupervised learning approaches are useful for successfully grouping postal service zones, according to our findings. Furthermore, by comparing a postal delivery region to other areas in the same group, workload balancing was achieved.

Creating a Smartphone User Recommendation System Using Clustering (클러스터링을 이용한 스마트폰 사용자 추천 시스템 만들기)

  • Jin Hyoung AN
    • Journal of Korea Artificial Intelligence Association
    • /
    • v.2 no.1
    • /
    • pp.1-6
    • /
    • 2024
  • In this paper, we develop an AI-based recommendation system that matches the specifications of smartphones from company 'S'. The system aims to simplify the complex decision-making process of consumers and guide them to choose the smartphone that best suits their daily needs. The recommendation system analyzes five specifications of smartphones (price, battery capacity, weight, camera quality, capacity) to help users make informed decisions without searching for extensive information. This approach not only saves time but also improves user satisfaction by ensuring that the selected smartphone closely matches the user's lifestyle and needs. The system utilizes unsupervised learning, i.e. clustering (K-MEANS, DBSCAN, Hierarchical Clustering), and provides personalized recommendations by evaluating them with silhouette scores, ensuring accurate and reliable grouping of similar smartphone models. By leveraging advanced data analysis techniques, the system can identify subtle patterns and preferences that might not be immediately apparent to consumers, enhancing the overall user experience. The ultimate goal of this AI recommendation system is to simplify the smartphone selection process, making it more accessible and user-friendly for all consumers. This paper discusses the data collection, preprocessing, development, implementation, and potential impact of the system using Pandas, crawling, scikit-learn, etc., and highlights the benefits of helping consumers explore the various options available and confidently choose the smartphone that best suits their daily lives.

Improvement of Self Organizing Maps using Gap Statistic and Probability Distribution

  • Jun, Sung-Hae
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.8 no.2
    • /
    • pp.116-120
    • /
    • 2008
  • Clustering is a method for unsupervised learning. General clustering tools have been depended on statistical methods and machine learning algorithms. One of the popular clustering algorithms based on machine learning is the self organizing map(SOM). SOM is a neural networks model for clustering. SOM and extended SOM have been used in diverse classification and clustering fields such as data mining. But, SOM has had a problem determining optimal number of clusters. In this paper, we propose an improvement of SOM using gap statistic and probability distribution. The gap statistic was introduced to estimate the number of clusters in a dataset. We use gap statistic for settling the problem of SOM. Also, in our research, weights of feature nodes are updated by probability distribution. After complete updating according to prior and posterior distributions, the weights of SOM have probability distributions for optima clustering. To verify improved performance of our work, we make experiments compared with other learning algorithms using simulation data sets.

A Clustering-based Semi-Supervised Learning through Initial Prediction of Unlabeled Data (미분류 데이터의 초기예측을 통한 군집기반의 부분지도 학습방법)

  • Kim, Eung-Ku;Jun, Chi-Hyuck
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.33 no.3
    • /
    • pp.93-105
    • /
    • 2008
  • Semi-supervised learning uses a small amount of labeled data to predict labels of unlabeled data as well as to improve clustering performance, whereas unsupervised learning analyzes only unlabeled data for clustering purpose. We propose a new clustering-based semi-supervised learning method by reflecting the initial predicted labels of unlabeled data on the objective function. The initial prediction should be done in terms of a discrete probability distribution through a classification method using labeled data. As a result, clusters are formed and labels of unlabeled data are predicted according to the Information of labeled data in the same cluster. We evaluate and compare the performance of the proposed method in terms of classification errors through numerical experiments with blinded labeled data.

Determining the Optimal Number of Signal Clusters Using Iterative HMM Classification

  • Ernest, Duker Junior;Kim, Yoon Joong
    • International journal of advanced smart convergence
    • /
    • v.7 no.2
    • /
    • pp.33-37
    • /
    • 2018
  • In this study, we propose an iterative clustering algorithm that automatically clusters a set of voice signal data without a label into an optimal number of clusters and generates hmm model for each cluster. In the clustering process, the likelihood calculations of the clusters are performed using iterative hmm learning and testing while varying the number of clusters for given data, and the maximum likelihood estimation method is used to determine the optimal number of clusters. We tested the effectiveness of this clustering algorithm on a small-vocabulary digit clustering task by mapping the unsupervised decoded output of the optimal cluster to the ground-truth transcription, we found out that they were highly correlated.

Identification of Plastic Wastes by Using Fuzzy Radial Basis Function Neural Networks Classifier with Conditional Fuzzy C-Means Clustering

  • Roh, Seok-Beom;Oh, Sung-Kwun
    • Journal of Electrical Engineering and Technology
    • /
    • v.11 no.6
    • /
    • pp.1872-1879
    • /
    • 2016
  • The techniques to recycle and reuse plastics attract public attention. These public attraction and needs result in improving the recycling technique. However, the identification technique for black plastic wastes still have big problem that the spectrum extracted from near infrared radiation spectroscopy is not clear and is contaminated by noise. To overcome this problem, we apply Raman spectroscopy to extract a clear spectrum of plastic material. In addition, to improve the classification ability of fuzzy Radial Basis Function Neural Networks, we apply supervised learning based clustering method instead of unsupervised clustering method. The conditional fuzzy C-Means clustering method, which is a kind of supervised learning based clustering algorithms, is used to determine the location of radial basis functions. The conditional fuzzy C-Means clustering analyzes the data distribution over input space under the supervision of auxiliary information. The auxiliary information is defined by using k Nearest Neighbor approach.