• Title/Summary/Keyword: k-means clustering analysis

Search Result 462, Processing Time 0.025 seconds

A Implementation of Optimal Multiple Classification System using Data Mining for Genome Analysis

  • Jeong, Yu-Jeong;Choi, Gwang-Mi
    • Journal of the Korea Society of Computer and Information
    • /
    • v.23 no.12
    • /
    • pp.43-48
    • /
    • 2018
  • In this paper, more efficient classification result could be obtained by applying the combination of the Hidden Markov Model and SVM Model to HMSV algorithm gene expression data which simulated the stochastic flow of gene data and clustering it. In this paper, we verified the HMSV algorithm that combines independently learned algorithms. To prove that this paper is superior to other papers, we tested the sensitivity and specificity of the most commonly used classification criteria. As a result, the K-means is 71% and the SOM is 68%. The proposed HMSV algorithm is 85%. These results are stable and high. It can be seen that this is better classified than using a general classification algorithm. The algorithm proposed in this paper is a stochastic modeling of the generation process of the characteristics included in the signal, and a good recognition rate can be obtained with a small amount of calculation, so it will be useful to study the relationship with diseases by showing fast and effective performance improvement with an algorithm that clusters nodes by simulating the stochastic flow of Gene Data through data mining of BigData.

Identifying Cluster Patterns in Relationship Between Municipal Revenue Configuration and Fiscal Surplus: Application of Machine Learning Methodologies

  • Im Chunghyeok;Ryou Jaemin;Han JunHyun;Bae Jayon
    • International Journal of Advanced Culture Technology
    • /
    • v.12 no.3
    • /
    • pp.159-164
    • /
    • 2024
  • Net surplus serves as a crucial indicator of how efficiently local governments utilize their resources. This study aims to analyze and categorize the patterns of net surplus across 75 local governments in Korea. By employing machine learning techniques such as K-means clustering and silhouette analysis, this research delves into surplus patterns, revealing insights that differ from those provided by traditional analytical methods. Machine learning enables a broader spectrum of discoveries, leading us to identify three distinct clusters in the net surplus of Korean local finances. The characteristics of these three clusters show that the wealthiest cities have the highest surplus ratios. In contrast, mid-sized municipalities, constrained by limited central government support and scarce local resources, exhibit the lowest surplus ratios. Interestingly, a significant number of cities maintain a median surplus ratio even under challenging fiscal conditions. Additionally, we identify critical thresholds that differentiate the three clusters: a grant-in-aid ratio of 19.31%, a debt ratio of 3.52%, and a local tax ratio of 25.58%. This identification of thresholds is a key contribution of our study, as these specific thresholds have not been previously addressed in the literature.

Genetic Diversity Based on Morphology and RAPD Analysis in Vegetable Soybean

  • Srinives, P.;Chowdhury, A.K.;Tongpamnak, P.;Saksoong, P.
    • KOREAN JOURNAL OF CROP SCIENCE
    • /
    • v.46 no.2
    • /
    • pp.112-120
    • /
    • 2001
  • Genetic diversity of 47 East-Asian vegetable soybean was characterized by means of agro-morphological traits and RAPD markers. A field trial was conducted to evaluate 14 agro-morphological traits. To study RAPD-based DNA analysis, a total of sixty 10-mer random primers were screened. Of these, 23 polymorphic markers in 16 varieties used for screening. Among 207 markers amplified, 48 were polymorphic for at least one pairwise comparison within the 47 varieties. A higher differentiation level between varieties was observed by using RAPD markers compared to morphological markers. Correspondence analysis using both types of marker showed that RAPD data could fully discriminate between all varieties, whereas morphological markers could not achieve a complete discrimination. Genetic distances between the varieties were estimated from simple matching coefficients, ranged from 0.0 to 0.640 with an average of 0.295$\pm$0.131 for morphological traits and 0.042 to 0.625 with an average of 0.336$\pm$0.099 for RAPD data, respectively. Cluster analysis based on genetic dissimilarity of these varieties gave rise to 4 distinct groups. The clustering results based on RAPDs did not match with those based on morphological traits. Geographical distribution of most varieties in each of the groups were not well defined. The results suggested that the level of genetic diversity within this group of East-Asian vegetable soybean varieties was sufficient for a breeding program and can be used to establish genetic relationships among them with unknown or unrelated pedigrees.

  • PDF

Anomaly Detection in Sensor Data

  • Kim, Jong-Min;Baik, Jaiwook
    • Journal of Applied Reliability
    • /
    • v.18 no.1
    • /
    • pp.20-32
    • /
    • 2018
  • Purpose: The purpose of this study is to set up an anomaly detection criteria for sensor data coming from a motorcycle. Methods: Five sensor values for accelerator pedal, engine rpm, transmission rpm, gear and speed are obtained every 0.02 second from a motorcycle. Exploratory data analysis is used to find any pattern in the data. Traditional process control methods such as X control chart and time series models are fitted to find any anomaly behavior in the data. Finally unsupervised learning algorithm such as k-means clustering is used to find any anomaly spot in the sensor data. Results: According to exploratory data analysis, the distribution of accelerator pedal sensor values is very much skewed to the left. The motorcycle seemed to have been driven in a city at speed less than 45 kilometers per hour. Traditional process control charts such as X control chart fail due to severe autocorrelation in each sensor data. However, ARIMA model found three abnormal points where they are beyond 2 sigma limits in the control chart. We applied a copula based Markov chain to perform statistical process control for correlated observations. Copula based Markov model found anomaly behavior in the similar places as ARIMA model. In an unsupervised learning algorithm, large sensor values get subdivided into two, three, and four disjoint regions. So extreme sensor values are the ones that need to be tracked down for any sign of anomaly behavior in the sensor values. Conclusion: Exploratory data analysis is useful to find any pattern in the sensor data. Process control chart using ARIMA and Joe's copula based Markov model also give warnings near similar places in the data. Unsupervised learning algorithm shows us that the extreme sensor values are the ones that need to be tracked down for any sign of anomaly behavior.

Design of Digit Recognition System Realized with the Aid of Fuzzy RBFNNs and Incremental-PCA (퍼지 RBFNNs와 증분형 주성분 분석법으로 실현된 숫자 인식 시스템의 설계)

  • Kim, Bong-Youn;Oh, Sung-Kwun;Kim, Jin-Yul
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.26 no.1
    • /
    • pp.56-63
    • /
    • 2016
  • In this study, we introduce a design of Fuzzy RBFNNs-based digit recognition system using the incremental-PCA in order to recognize the handwritten digits. The Principal Component Analysis (PCA) is a widely-adopted dimensional reduction algorithm, but it needs high computing overhead for feature extraction in case of using high dimensional images or a large amount of training data. To alleviate such problem, the incremental-PCA is proposed for the computationally efficient processing as well as the incremental learning of high dimensional data in the feature extraction stage. The architecture of Fuzzy Radial Basis Function Neural Networks (RBFNN) consists of three functional modules such as condition, conclusion, and inference part. In the condition part, the input space is partitioned with the use of fuzzy clustering realized by means of the Fuzzy C-Means (FCM) algorithm. Also, it is used instead of gaussian function to consider the characteristic of input data. In the conclusion part, connection weights are used as the extended diverse types in polynomial expression such as constant, linear, quadratic and modified quadratic. Experimental results conducted on the benchmarking MNIST handwritten digit database demonstrate the effectiveness and efficiency of the proposed digit recognition system when compared with other studies.

Multivariate Stratification Method for the Multipurpose Sample Survey : A Case Study of the Sample Design for Fisher Production Survey (다목적 표본조사를 위한 다변량 층화 : 어업비계통생산량조사를 위한 표본설계 사례)

  • Park, Jin-Woo;Kim, Young-Won;Lee, Seok-Hoon;Shin, Ji-Eun
    • Survey Research
    • /
    • v.9 no.1
    • /
    • pp.69-85
    • /
    • 2008
  • Stratification is a feature of the majority of field sample design. This paper considers the multivariate stratification strategy for multipurpose sample survey with several auxiliary variables. In a multipurpose survey, stratification procedure is very complicated because we have to simultaneously consider the efficiencies of stratification for several variables of interest. We propose stratification strategy based on factor analysis and cluster analysis using several stratification variables. To improve the efficiency of stratification, we first select the stratification variables by factor analysis, and then apply the K-means clustering algorithm to the formation of strata. An application of the stratification strategy in the sampling design for the Fisher Production Survey is discussed, and it turns out that the variances of estimators are significantly less than those obtained by simple random sampling.

  • PDF

Trend Analysis of Grow-Your-Own Using Social Network Analysis: Focusing on Hashtags on Instagram

  • Park, Yumin;Shin, Yong-Wook
    • Journal of People, Plants, and Environment
    • /
    • v.24 no.5
    • /
    • pp.451-460
    • /
    • 2021
  • Background and objective: The prolonged COVID-19 pandemic has had significant impacts on mental health, which has emerged as a major public health issue around the world. This study aimed to analyze trends and network structure of 'grow-your-own (GYO)' through Instagram, one of the most influential social media platforms, to encourage and sustain home gardening activities for promotion of emotional support and physical health. Methods: A total of 6,388 posts including keyword hashtags '#gyo' and '#growyourown' on Instagram from June 13, 2020 to April 13, 2021 were collected. Word embedding was performed using Word2Vec library, and 7 clusters were identified with K-means clustering: GYO, garden and gardening, allotment, kitchen garden, sustainability, urban gardening, etc. Moreover, we conducted social network analysis to determine the centrality of related words and visualized the results using Gephi 0.9.2. Results: The analysis showed that various combinations of words, such as #growourrownfood, #growourrownveggies, and #growwhatyoueat revealed preference and interest of users in GYO, and appeared to encourage their activities on Instagram. In particular, #gardeningtips, #greenfingers, #goodlife, #gardeninglife, #gardensofinstagram were found to express positive emotions and pride as a gardener by sharing their daily gardening lives. Users were participating in urban gardening through #allotment, #raisedbeds, #kitchengarden and we could identify trends toward self-sufficiency and sustainable living. Conclusion: Based on these findings, it is expected that the trend data of GYO, which is a form of urban gardening, can be used as the basic data to establish urban gardening plans considering each characteristic, such as the emotions and identity of participants as well as their dispositions.

Linear interpolation and Machine Learning Methods for Gas Leakage Prediction Base on Multi-source Data Integration (다중소스 데이터 융합 기반의 가스 누출 예측을 위한 선형 보간 및 머신러닝 기법)

  • Dashdondov, Khongorzul;Jo, Kyuri;Kim, Mi-Hye
    • Journal of the Korea Convergence Society
    • /
    • v.13 no.3
    • /
    • pp.33-41
    • /
    • 2022
  • In this article, we proposed to predict natural gas (NG) leakage levels through feature selection based on a factor analysis (FA) of the integrating the Korean Meteorological Agency data and natural gas leakage data for considering complex factors. The paper has been divided into three modules. First, we filled missing data based on the linear interpolation method on the integrated data set, and selected essential features using FA with OrdinalEncoder (OE)-based normalization. The dataset is labeled by K-means clustering. The final module uses four algorithms, K-nearest neighbors (KNN), decision tree (DT), random forest (RF), Naive Bayes (NB), to predict gas leakage levels. The proposed method is evaluated by the accuracy, area under the ROC curve (AUC), and mean standard error (MSE). The test results indicate that the OrdinalEncoder-Factor analysis (OE-F)-based classification method has improved successfully. Moreover, OE-F-based KNN (OE-F-KNN) showed the best performance by giving 95.20% accuracy, an AUC of 96.13%, and an MSE of 0.031.

A Statistical Approach for Improving the Embedding Capacity of Block Matching based Image Steganography (블록 매칭 기반 영상 스테가노그래피의 삽입 용량 개선을 위한 통계적 접근 방법)

  • Kim, Jaeyoung;Park, Hanhoon;Park, Jong-Il
    • Journal of Broadcast Engineering
    • /
    • v.22 no.5
    • /
    • pp.643-651
    • /
    • 2017
  • Steganography is one of information hiding technologies and discriminated from cryptography in that it focuses on avoiding the existence the hidden information from being detected by third parties, rather than protecting it from being decoded. In this paper, as an image steganography method which uses images as media, we propose a new block matching method that embeds information into the discrete wavelet transform (DWT) domain. The proposed method, based on a statistical analysis, reduces loss of embedding capacity due to inequable use of candidate blocks. It works in such a way that computes the variance of each candidate block, preserves candidate blocks with high frequency components while reducing candidate blocks with low frequency components by compressing them exploiting the k-means clustering algorithm. Compared with the previous block matching method, the proposed method can reconstruct secret images with similar PSNRs while embedding higher-capacity information.

Online news-based stock price forecasting considering homogeneity in the industrial sector (산업군 내 동질성을 고려한 온라인 뉴스 기반 주가예측)

  • Seong, Nohyoon;Nam, Kihwan
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.2
    • /
    • pp.1-19
    • /
    • 2018
  • Since stock movements forecasting is an important issue both academically and practically, studies related to stock price prediction have been actively conducted. The stock price forecasting research is classified into structured data and unstructured data, and it is divided into technical analysis, fundamental analysis and media effect analysis in detail. In the big data era, research on stock price prediction combining big data is actively underway. Based on a large number of data, stock prediction research mainly focuses on machine learning techniques. Especially, research methods that combine the effects of media are attracting attention recently, among which researches that analyze online news and utilize online news to forecast stock prices are becoming main. Previous studies predicting stock prices through online news are mostly sentiment analysis of news, making different corpus for each company, and making a dictionary that predicts stock prices by recording responses according to the past stock price. Therefore, existing studies have examined the impact of online news on individual companies. For example, stock movements of Samsung Electronics are predicted with only online news of Samsung Electronics. In addition, a method of considering influences among highly relevant companies has also been studied recently. For example, stock movements of Samsung Electronics are predicted with news of Samsung Electronics and a highly related company like LG Electronics.These previous studies examine the effects of news of industrial sector with homogeneity on the individual company. In the previous studies, homogeneous industries are classified according to the Global Industrial Classification Standard. In other words, the existing studies were analyzed under the assumption that industries divided into Global Industrial Classification Standard have homogeneity. However, existing studies have limitations in that they do not take into account influential companies with high relevance or reflect the existence of heterogeneity within the same Global Industrial Classification Standard sectors. As a result of our examining the various sectors, it can be seen that there are sectors that show the industrial sectors are not a homogeneous group. To overcome these limitations of existing studies that do not reflect heterogeneity, our study suggests a methodology that reflects the heterogeneous effects of the industrial sector that affect the stock price by applying k-means clustering. Multiple Kernel Learning is mainly used to integrate data with various characteristics. Multiple Kernel Learning has several kernels, each of which receives and predicts different data. To incorporate effects of target firm and its relevant firms simultaneously, we used Multiple Kernel Learning. Each kernel was assigned to predict stock prices with variables of financial news of the industrial group divided by the target firm, K-means cluster analysis. In order to prove that the suggested methodology is appropriate, experiments were conducted through three years of online news and stock prices. The results of this study are as follows. (1) We confirmed that the information of the industrial sectors related to target company also contains meaningful information to predict stock movements of target company and confirmed that machine learning algorithm has better predictive power when considering the news of the relevant companies and target company's news together. (2) It is important to predict stock movements with varying number of clusters according to the level of homogeneity in the industrial sector. In other words, when stock prices are homogeneous in industrial sectors, it is important to use relational effect at the level of industry group without analyzing clusters or to use it in small number of clusters. When the stock price is heterogeneous in industry group, it is important to cluster them into groups. This study has a contribution that we testified firms classified as Global Industrial Classification Standard have heterogeneity and suggested it is necessary to define the relevance through machine learning and statistical analysis methodology rather than simply defining it in the Global Industrial Classification Standard. It has also contribution that we proved the efficiency of the prediction model reflecting heterogeneity.