• Title/Summary/Keyword: optimal number of clusters

Search Result 79, Processing Time 0.03 seconds

Classification of Weather Patterns in the East Asia Region using the K-means Clustering Analysis (K-평균 군집분석을 이용한 동아시아 지역 날씨유형 분류)

  • Cho, Young-Jun;Lee, Hyeon-Cheol;Lim, Byunghwan;Kim, Seung-Bum
    • Atmosphere
    • /
    • v.29 no.4
    • /
    • pp.451-461
    • /
    • 2019
  • Medium-range forecast is highly dependent on ensemble forecast data. However, operational weather forecasters have not enough time to digest all of detailed features revealed in ensemble forecast data. To utilize the ensemble data effectively in medium-range forecasting, representative weather patterns in East Asia in this study are defined. The k-means clustering analysis is applied for the objectivity of weather patterns. Input data used daily Mean Sea Level Pressure (MSLP) anomaly of the ECMWF ReAnalysis-Interim (ERA-Interim) during 1981~2010 (30 years) provided by the European Centre for Medium-Range Weather Forecasts (ECMWF). Using the Explained Variance (EV), the optimal study area is defined by 20~60°N, 100~150°E. The number of clusters defined by Explained Cluster Variance (ECV) is thirty (k = 30). 30 representative weather patterns with their frequencies are summarized. Weather pattern #1 occurred all seasons, but it was about 56% in summer (June~September). The relatively rare occurrence of weather pattern (#30) occurred mainly in winter. Additionally, we investigate the relationship between weather patterns and extreme weather events such as heat wave, cold wave, and heavy rainfall as well as snowfall. The weather patterns associated with heavy rainfall exceeding 110 mm day-1 were #1, #4, and #9 with days (%) of more than 10%. Heavy snowfall events exceeding 24 cm day-1 mainly occurred in weather pattern #28 (4%) and #29 (6%). High and low temperature events (> 34℃ and < -14℃) were associated with weather pattern #1~4 (14~18%) and #28~29 (27~29%), respectively. These results suggest that the classification of various weather patterns will be used as a reference for grouping all ensemble forecast data, which will be useful for the scenario-based medium-range ensemble forecast in the future.

An Optimal Cluster Analysis Method with Fuzzy Performance Measures (퍼지 성능 측정자를 결합한 최적 클러스터 분석방법)

  • 이현숙;오경환
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.6 no.3
    • /
    • pp.81-88
    • /
    • 1996
  • Cluster analysis is based on partitioning a collection of data points into a number of clusters, where the data points in side a cluster have a certain degree of similarity and it is a fundamental process of data analysis. So, it has been playing an important role in solving many problems in pattern recognition and image processing. For these many clustering algorithms depending on distance criteria have been developed and fuzzy set theory has been introduced to reflect the description of real data, where boundaries might be fuzzy. If fuzzy cluster analysis is tomake a significant contribution to engineering applications, much more attention must be paid to fundamental questions of cluster validity problem which is how well it has identified the structure that is present in the data. Several validity functionals such as partition coefficient, claasification entropy and proportion exponent, have been used for measuring validity mathematically. But the issue of cluster validity involves complex aspects, it is difficult to measure it with one measuring function as the conventional study. In this paper, we propose four performance indices and the way to measure the quality of clustering formed by given learning strategy.

  • PDF

A new cluster validity index based on connectivity in self-organizing map (자기조직화지도에서 연결강도에 기반한 새로운 군집타당성지수)

  • Kim, Sangmin;Kim, Jaejik
    • The Korean Journal of Applied Statistics
    • /
    • v.33 no.5
    • /
    • pp.591-601
    • /
    • 2020
  • The self-organizing map (SOM) is a unsupervised learning method projecting high-dimensional data into low-dimensional nodes. It can visualize data in 2 or 3 dimensional space using the nodes and it is available to explore characteristics of data through the nodes. To understand the structure of data, cluster analysis is often used for nodes obtained from SOM. In cluster analysis, the optimal number of clusters is one of important issues. To help to determine it, various cluster validity indexes have been developed and they can be applied to clustering outcomes for nodes from SOM. However, while SOM has an advantage in that it reflects the topological properties of original data in the low-dimensional space, these indexes do not consider it. Thus, we propose a new cluster validity index for SOM based on connectivity between nodes which considers topological properties of data. The performance of the proposed index is evaluated through simulations and it is compared with various existing cluster validity indexes.

The Application of Genetic Algorithm for the Identification of Discontinuity Sets (불연속면 군 분류를 위한 유전자알고리즘의 응용)

  • Sunwoo Choon;Jung Yong-Bok
    • Tunnel and Underground Space
    • /
    • v.15 no.1 s.54
    • /
    • pp.47-54
    • /
    • 2005
  • One of the standard procedures of discontinuity survey is the joint set identification from the population of field orientation data. Discontinuity set identification is fundamental to rock engineering tasks such as rock mass classification, discrete element analysis, key block analysis. and discrete fracture network modeling. Conventionally, manual method using contour plot had been widely used for this task, but this method has some short-comings such as yielding subjective identification results, manual operations, and so on. In this study, the method of discontinuity set identification using genetic algorithm was introduced, but slightly modified to handle the orientation data. Finally, based on the genetic algorithm, we developed a FORTRAN program, Genetic Algorithm based Clustering(GAC) and applied it to two different discontinuity data sets. Genetic Algorithm based Clustering(GAC) was proved to be a fast and efficient method for the discontinuity set identification task. In addition, fitness function based on variance showed more efficient performance in finding the optimal number of clusters when compared with Davis - Bouldin index.

Breeding and characterization of a new white cultivar of Pleurotus ostreatus, 'Sena' (갓이 백색인 느타리 신품종 '세나'의 육성 및 특성)

  • Minji Oh;Min-Sik Kim;Ji-Hoon Im;Youn-Lee Oh
    • Journal of Mushroom
    • /
    • v.21 no.3
    • /
    • pp.179-184
    • /
    • 2023
  • The development of automated bottle cultivation systems has facilitated the large-scale production of Pleurotus ostreatus, a commonly cultivated oyster mushroom species in South Korea. However, as the consumption of this product is decreasing and production quantities are exceeding demand, farmers are seeking various other mushroom types and cultivars. In response to this, we have developed a new oyster mushroom cultivar named 'Sena'. This high-yielding cultivar has a white pileus and excellent quality. The white oyster mushroom cultivars 'Goni' and 'Miso' were selected as parental strains from the genetic resources of the National Institute of Horticultural and Herbal Science's Mushroom Division. By crossing their monokaryons, hybrids were developed and subjected to cultivation trials and characteristic evaluations to select the superior cultivar. The optimal temperature for 'Sena' mycelial growth is 25-30℃, with inhibition occurring at temperatures above 30℃, whereas the temperature for mushroom growth is 14-18℃. The mushrooms grow in clusters, with the white pileus having a shallow funnel shape. Optimal mycelial growth occurs in malt extract agar medium. When cultivated in 1,100 cc bottles, the 'Sena' cultivar had 35 available individuals, surpassing the number 16 available from the control cultivar 'Goni'. The yield per bottle also increased by approximately 157 g, a 24% increase over the control cultivar amount. When 300 g samples of harvested mushrooms were packed and stored at 4℃ in a cold storage facility for 28 days, the weight loss rate of 'Sena' was approximately 4.22%, lower than that of 'Goni'. Moreover, the changes in pileus and stipe whiteness (measuring 6.99 and 8.33, respectively) were also lower than those of the control cultivar. Since the appearance of a white cap is crucial for quality assessment, the 'Sena' cultivar is superior to the 'Goni' cultivar in terms of both weight and quality after undergoing low-temperature storage.

Construction and Application of Network Design System for Optimal Water Quality Monitoring in Reservoir (저수지 최적수질측정망 구축시스템 개발 및 적용)

  • Lee, Yo-Sang;Kwon, Se-Hyug;Lee, Sang-Uk;Ban, Yang-Jin
    • Journal of Korea Water Resources Association
    • /
    • v.44 no.4
    • /
    • pp.295-304
    • /
    • 2011
  • For effective water quality management, it is necessary to secure reliable water quality information. There are many variables that need to be included in a comprehensive practical monitoring network : representative sampling locations, suitable sampling frequencies, water quality variable selection, and budgetary and logistical constraints are examples, especially sampling location is considered to be the most important issues. Until now, monitoring network design for water quality management was set according to the qualitative judgments, which is a problem of representativeness. In this paper, we propose network design system for optimal water quality monitoring using the scientific statistical techniques. Network design system is made based on the SAS program of version 9.2 and configured with simple input system and user friendly outputs considering the convenience of users. It applies to Excel data format for ease to use and all data of sampling location is distinguished to sheet base. In this system, time plots, dendrogram, and scatter plots are shown as follows: Time plots of water quality variables are graphed for identifying variables to classify sampling locations significantly. Similarities of sampling locations are calculated using euclidean distances of principal component variables and dimension coordinate of multidimensional scaling method are calculated and dendrogram by clustering analysis is represented and used for users to choose an appropriate number of clusters. Scatter plots of principle component variables are shown for clustering information with sampling locations and representative location.

Predictive Clustering-based Collaborative Filtering Technique for Performance-Stability of Recommendation System (추천 시스템의 성능 안정성을 위한 예측적 군집화 기반 협업 필터링 기법)

  • Lee, O-Joun;You, Eun-Soon
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.1
    • /
    • pp.119-142
    • /
    • 2015
  • With the explosive growth in the volume of information, Internet users are experiencing considerable difficulties in obtaining necessary information online. Against this backdrop, ever-greater importance is being placed on a recommender system that provides information catered to user preferences and tastes in an attempt to address issues associated with information overload. To this end, a number of techniques have been proposed, including content-based filtering (CBF), demographic filtering (DF) and collaborative filtering (CF). Among them, CBF and DF require external information and thus cannot be applied to a variety of domains. CF, on the other hand, is widely used since it is relatively free from the domain constraint. The CF technique is broadly classified into memory-based CF, model-based CF and hybrid CF. Model-based CF addresses the drawbacks of CF by considering the Bayesian model, clustering model or dependency network model. This filtering technique not only improves the sparsity and scalability issues but also boosts predictive performance. However, it involves expensive model-building and results in a tradeoff between performance and scalability. Such tradeoff is attributed to reduced coverage, which is a type of sparsity issues. In addition, expensive model-building may lead to performance instability since changes in the domain environment cannot be immediately incorporated into the model due to high costs involved. Cumulative changes in the domain environment that have failed to be reflected eventually undermine system performance. This study incorporates the Markov model of transition probabilities and the concept of fuzzy clustering with CBCF to propose predictive clustering-based CF (PCCF) that solves the issues of reduced coverage and of unstable performance. The method improves performance instability by tracking the changes in user preferences and bridging the gap between the static model and dynamic users. Furthermore, the issue of reduced coverage also improves by expanding the coverage based on transition probabilities and clustering probabilities. The proposed method consists of four processes. First, user preferences are normalized in preference clustering. Second, changes in user preferences are detected from review score entries during preference transition detection. Third, user propensities are normalized using patterns of changes (propensities) in user preferences in propensity clustering. Lastly, the preference prediction model is developed to predict user preferences for items during preference prediction. The proposed method has been validated by testing the robustness of performance instability and scalability-performance tradeoff. The initial test compared and analyzed the performance of individual recommender systems each enabled by IBCF, CBCF, ICFEC and PCCF under an environment where data sparsity had been minimized. The following test adjusted the optimal number of clusters in CBCF, ICFEC and PCCF for a comparative analysis of subsequent changes in the system performance. The test results revealed that the suggested method produced insignificant improvement in performance in comparison with the existing techniques. In addition, it failed to achieve significant improvement in the standard deviation that indicates the degree of data fluctuation. Notwithstanding, it resulted in marked improvement over the existing techniques in terms of range that indicates the level of performance fluctuation. The level of performance fluctuation before and after the model generation improved by 51.31% in the initial test. Then in the following test, there has been 36.05% improvement in the level of performance fluctuation driven by the changes in the number of clusters. This signifies that the proposed method, despite the slight performance improvement, clearly offers better performance stability compared to the existing techniques. Further research on this study will be directed toward enhancing the recommendation performance that failed to demonstrate significant improvement over the existing techniques. The future research will consider the introduction of a high-dimensional parameter-free clustering algorithm or deep learning-based model in order to improve performance in recommendations.

Prognostic Evaluation of Categorical Platelet-based Indices Using Clustering Methods Based on the Monte Carlo Comparison for Hepatocellular Carcinoma

  • Guo, Pi;Shen, Shun-Li;Zhang, Qin;Zeng, Fang-Fang;Zhang, Wang-Jian;Hu, Xiao-Min;Zhang, Ding-Mei;Peng, Bao-Gang;Hao, Yuan-Tao
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.15 no.14
    • /
    • pp.5721-5727
    • /
    • 2014
  • Objectives: To evaluate the performance of clustering methods used in the prognostic assessment of categorical clinical data for hepatocellular carcinoma (HCC) patients in China, and establish a predictable prognostic nomogram for clinical decisions. Materials and Methods: A total of 332 newly diagnosed HCC patients treated with hepatic resection during 2006-2009 were enrolled. Patients were regularly followed up at outpatient clinics. Clustering methods including the Average linkage, k-modes, fuzzy k-modes, PAM, CLARA, protocluster, and ROCK were compared by Monte Carlo simulation, and the optimal method was applied to investigate the clustering pattern of the indices including platelet count, platelet/lymphocyte ratio (PLR) and serum aspartate aminotransferase activity/platelet count ratio index (APRI). Then the clustering variable, age group, tumor size, number of tumor and vascular invasion were studied in a multivariable Cox regression model. A prognostic nomogram was constructed for clinical decisions. Results: The ROCK was best in both the overlapping and non-overlapping cases performed to assess the prognostic value of platelet-based indices. Patients with categorical platelet-based indices significantly split across two clusters, and those with high values, had a high risk of HCC recurrence (hazard ratio [HR] 1.42, 95% CI 1.09-1.86; p<0.01). Tumor size, number of tumor and blood vessel invasion were also associated with high risk of HCC recurrence (all p< 0.01). The nomogram well predicted HCC patient survival at 3 and 5 years. Conclusions: A cluster of platelet-based indices combined with other clinical covariates could be used for prognosis evaluation in HCC.

UX Methodology Study by Data Analysis Focusing on deriving persona through customer segment classification (데이터 분석을 통한 UX 방법론 연구 고객 세그먼트 분류를 통한 페르소나 도출을 중심으로)

  • Lee, Seul-Yi;Park, Do-Hyung
    • Journal of Intelligence and Information Systems
    • /
    • v.27 no.1
    • /
    • pp.151-176
    • /
    • 2021
  • As the information technology industry develops, various kinds of data are being created, and it is now essential to process them and use them in the industry. Analyzing and utilizing various digital data collected online and offline is a necessary process to provide an appropriate experience for customers in the industry. In order to create new businesses, products, and services, it is essential to use customer data collected in various ways to deeply understand potential customers' needs and analyze behavior patterns to capture hidden signals of desire. However, it is true that research using data analysis and UX methodology, which should be conducted in parallel for effective service development, is being conducted separately and that there is a lack of examples of use in the industry. In thiswork, we construct a single process by applying data analysis methods and UX methodologies. This study is important in that it is highly likely to be used because it applies methodologies that are actively used in practice. We conducted a survey on the topic to identify and cluster the associations between factors to establish customer classification and target customers. The research methods are as follows. First, we first conduct a factor, regression analysis to determine the association between factors in the happiness data survey. Groups are grouped according to the survey results and identify the relationship between 34 questions of psychological stability, family life, relational satisfaction, health, economic satisfaction, work satisfaction, daily life satisfaction, and residential environment satisfaction. Second, we classify clusters based on factors affecting happiness and extract the optimal number of clusters. Based on the results, we cross-analyzed the characteristics of each cluster. Third, forservice definition, analysis was conducted by correlating with keywords related to happiness. We leverage keyword analysis of the thumb trend to derive ideas based on the interest and associations of the keyword. We also collected approximately 11,000 news articles based on the top three keywords that are highly related to happiness, then derived issues between keywords through text mining analysis in SAS, and utilized them in defining services after ideas were conceived. Fourth, based on the characteristics identified through data analysis, we selected segmentation and targetingappropriate for service discovery. To this end, the characteristics of the factors were grouped and selected into four groups, and the profile was drawn up and the main target customers were selected. Fifth, based on the characteristics of the main target customers, interviewers were selected and the In-depthinterviews were conducted to discover the causes of happiness, causes of unhappiness, and needs for services. Sixth, we derive customer behavior patterns based on segment results and detailed interviews, and specify the objectives associated with the characteristics. Seventh, a typical persona using qualitative surveys and a persona using data were produced to analyze each characteristic and pros and cons by comparing the two personas. Existing market segmentation classifies customers based on purchasing factors, and UX methodology measures users' behavior variables to establish criteria and redefine users' classification. Utilizing these segment classification methods, applying the process of producinguser classification and persona in UX methodology will be able to utilize them as more accurate customer classification schemes. The significance of this study is summarized in two ways: First, the idea of using data to create a variety of services was linked to the UX methodology used to plan IT services by applying it in the hot topic era. Second, we further enhance user classification by applying segment analysis methods that are not currently used well in UX methodologies. To provide a consistent experience in creating a single service, from large to small, it is necessary to define customers with common goals. To this end, it is necessary to derive persona and persuade various stakeholders. Under these circumstances, designing a consistent experience from beginning to end, through fast and concrete user descriptions, would be a very effective way to produce a successful service.