• Title/Summary/Keyword: K-Means 클러스터링

Search Result 363, Processing Time 0.037 seconds

Semi-automatic Construction of Learning Set and Integration of Automatic Classification for Academic Literature in Technical Sciences (기술과학 분야 학술문헌에 대한 학습집합 반자동 구축 및 자동 분류 통합 연구)

  • Kim, Seon-Wu;Ko, Gun-Woo;Choi, Won-Jun;Jeong, Hee-Seok;Yoon, Hwa-Mook;Choi, Sung-Pil
    • Journal of the Korean Society for information Management
    • /
    • v.35 no.4
    • /
    • pp.141-164
    • /
    • 2018
  • Recently, as the amount of academic literature has increased rapidly and complex researches have been actively conducted, researchers have difficulty in analyzing trends in previous research. In order to solve this problem, it is necessary to classify information in units of academic papers. However, in Korea, there is no academic database in which such information is provided. In this paper, we propose an automatic classification system that can classify domestic academic literature into multiple classes. To this end, first, academic documents in the technical science field described in Korean were collected and mapped according to class 600 of the DDC by using K-Means clustering technique to construct a learning set capable of multiple classification. As a result of the construction of the training set, 63,915 documents in the Korean technical science field were established except for the values in which metadata does not exist. Using this training set, we implemented and learned the automatic classification engine of academic documents based on deep learning. Experimental results obtained by hand-built experimental set-up showed 78.32% accuracy and 72.45% F1 performance for multiple classification.

Research on the Application of AI Techniques to Advance Dam Operation (댐 운영 고도화를 위한 AI 기법 적용 연구)

  • Choi, Hyun Gu;Jeong, Seok Il;Park, Jin Yong;Kwon, E Jae;Lee, Jun Yeol
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2022.05a
    • /
    • pp.387-387
    • /
    • 2022
  • 기존 홍수기시 댐 운영은 예측 강우와 실시간 관측 강우를 이용하여 댐 운영 모형을 수행하며, 예측 결과에 따라 의사결정 및 댐 운영을 실시하게 된다. 하지만 이 과정에서 반복적인 분석이 필요하며, 댐 운영 모형 수행자의 경험에 따라 예측 결과가 달라져서 반복작업에 대한 자동화, 모형 수행자에 따라 달라지지 않는 예측 결과의 일반화가 필요한 상황이다. 이에 댐 운영 모형에 AI 기법을 적용하여, 다양한 강우 상황에 따른 자동 예측 및 모형 결과의 일반화를 구현하고자 하였다. 이를 위해 수자원 분야에 적용된 국내외 129개 연구논문에서 사용된 딥러닝 기법의 활용성을 분석하였으며, 다양한 수자원 분야 AI 적용 사례 중에서 댐 운영 예측 모형에 적용한 사례는 없었지만 유사한 분야로는 장기 저수지 운영 예측과 댐 상·하류 수위, 유량 예측이 있었다. 수자원의 시계열 자료 활용을 위해서는 Long-Short Term Memory(LSTM) 기법의 적용 활용성이 높은 것으로 분석되었다. 댐 운영 모형에서 AI 적용은 2개 분야에서 진행하였다. 기존 강우관측소의 관측 강우를 활용하여 강우의 패턴분석을 수행하는 과정과, 강우에서 댐 유입량 산정시 매개변수 최적화 분야에 적용하였다. 강우 패턴분석에서는 유사한 표본끼리 묶음을 생성하는 K-means 클러스터링 알고리즘과 시계열 데이터의 유사도 분석 방법인 Dynamic Time Warping을 결합하여 적용하였다. 강우 패턴분석을 통해서 지점별로 월별, 태풍 및 장마기간에 가장 많이 관측되었던 강우 패턴을 제시하며, 이를 모형에서 직접적으로 활용할 수 있도록 구성하였다. 강우에서 댐 유입량을 산정시 활용되는 매개변수 최적화를 위해서는 3층의 Multi-Layer LSTM 기법과 경사하강법을 적용하였다. 매개변수 최적화에 적용되는 매개변수는 중권역별 8개이며, 매개변수 최적화 과정을 통해 산정되는 결과물은 실측값과 오차가 제일 적은 유량(유입량)이 된다. 댐 운영 모형에 AI 기법을 적용한 결과 기존 반복작업에 대한 자동화는 이뤘으며, 댐 운영에 따른 상·하류 제약사항 표출 기능을 추가하여 의사결정에 소요되는 시간도 많이 줄일 수 있었다. 하지만, 매개변수 최적화 부분에서 기존 댐운영 모형에 적용되어 있는 고전적인 매개변수 추정기법보다 추정시간이 오래 소요되며, 매개변수 추정결과의 일반화가 이뤄지지 않아 이 부분에 대한 추가적인 연구가 필요하다.

  • PDF

Korean Phoneme Recognition Using Self-Organizing Feature Map (SOFM 신경회로망을 이용한 한국어 음소 인식)

  • Jeon, Yong-Koo;Yang, Jin-Woo;Kim, Soon-Hyob
    • The Journal of the Acoustical Society of Korea
    • /
    • v.14 no.2
    • /
    • pp.101-112
    • /
    • 1995
  • In order to construct a feature map-based phoneme classification system for speech recognition, two procedures are usually required. One is clustering and the other is labeling. In this paper, we present a phoneme classification system based on the Kohonen's Self-Organizing Feature Map (SOFM) for clusterer and labeler. It is known that the SOFM performs self-organizing process by which optimal local topographical mapping of the signal space and yields a reasonably high accuracy in recognition tasks. Consequently, SOFM can effectively be applied to the recognition of phonemes. Besides to improve the performance of the phoneme classification system, we propose the learning algorithm combined with the classical K-mans clustering algorithm in fine-tuning stage. In order to evaluate the performance of the proposed phoneme classification algorithm, we first use totaly 43 phonemes which construct six intra-class feature maps for six different phoneme classes. From the speaker-dependent phoneme classification tests using these six feature maps, we obtain recognition rate of $87.2\%$ and confirm that the proposed algorithm is an efficient method for improvement of recognition performance and convergence speed.

  • PDF

Development of Drought Map Based on Three-dimensional Spatio-temporal Analysis of Drought (가뭄사상에 대한 3차원적 시공간 분석을 통한 가뭄지도 개발)

  • Yoo, Jiyoung;So, Byung-Jin;Kwon, Hyun-Han;Kim, Tae-Woong
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.40 no.1
    • /
    • pp.25-33
    • /
    • 2020
  • A drought event is characterized by duration, severity and affected area. In general, after calculating a drought index using hydro-meteorological time series at a station, a drought event is defined based on the run theory to identify the beginning and end time. However, this one-dimensional analysis has limitations for analyzing the spatio-temporal occurrence characteristics and movement paths of drought. Therefore, this study is to define a three-dimensional drought event using a simple clustering algorithm and to develop a drought map that can be used to understand the drought severity according to the spatio-temporal expansion of drought. As a result, compared with the two-dimensional monitoring information to show spatial distribution of drought index, a proposed drought map is able to show three-dimensional drought characteristics inclusing drought duration, spatial cumulative severity, and centroid of drought. The analysis of drought map indicated that there was a drought event which had the affected area less than 10 % while on occations while there were 11 drought events (44 %) which had the affected area more a than 90 % of the total area. This means that it is important to understand the relationship between spatial variation of drought affected area and severity corresponding to various drought durations. The development of drought map based on three-dimensional drought analysis is useful to analyze the spatio-temporal occurrence characteristics and propagation patterns of regional drought which can be utilized in developing mitigation measures for future extreme droughts.

User-Perspective Issue Clustering Using Multi-Layered Two-Mode Network Analysis (다계층 이원 네트워크를 활용한 사용자 관점의 이슈 클러스터링)

  • Kim, Jieun;Kim, Namgyu;Cho, Yoonho
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.2
    • /
    • pp.93-107
    • /
    • 2014
  • In this paper, we report what we have observed with regard to user-perspective issue clustering based on multi-layered two-mode network analysis. This work is significant in the context of data collection by companies about customer needs. Most companies have failed to uncover such needs for products or services properly in terms of demographic data such as age, income levels, and purchase history. Because of excessive reliance on limited internal data, most recommendation systems do not provide decision makers with appropriate business information for current business circumstances. However, part of the problem is the increasing regulation of personal data gathering and privacy. This makes demographic or transaction data collection more difficult, and is a significant hurdle for traditional recommendation approaches because these systems demand a great deal of personal data or transaction logs. Our motivation for presenting this paper to academia is our strong belief, and evidence, that most customers' requirements for products can be effectively and efficiently analyzed from unstructured textual data such as Internet news text. In order to derive users' requirements from textual data obtained online, the proposed approach in this paper attempts to construct double two-mode networks, such as a user-news network and news-issue network, and to integrate these into one quasi-network as the input for issue clustering. One of the contributions of this research is the development of a methodology utilizing enormous amounts of unstructured textual data for user-oriented issue clustering by leveraging existing text mining and social network analysis. In order to build multi-layered two-mode networks of news logs, we need some tools such as text mining and topic analysis. We used not only SAS Enterprise Miner 12.1, which provides a text miner module and cluster module for textual data analysis, but also NetMiner 4 for network visualization and analysis. Our approach for user-perspective issue clustering is composed of six main phases: crawling, topic analysis, access pattern analysis, network merging, network conversion, and clustering. In the first phase, we collect visit logs for news sites by crawler. After gathering unstructured news article data, the topic analysis phase extracts issues from each news article in order to build an article-news network. For simplicity, 100 topics are extracted from 13,652 articles. In the third phase, a user-article network is constructed with access patterns derived from web transaction logs. The double two-mode networks are then merged into a quasi-network of user-issue. Finally, in the user-oriented issue-clustering phase, we classify issues through structural equivalence, and compare these with the clustering results from statistical tools and network analysis. An experiment with a large dataset was performed to build a multi-layer two-mode network. After that, we compared the results of issue clustering from SAS with that of network analysis. The experimental dataset was from a web site ranking site, and the biggest portal site in Korea. The sample dataset contains 150 million transaction logs and 13,652 news articles of 5,000 panels over one year. User-article and article-issue networks are constructed and merged into a user-issue quasi-network using Netminer. Our issue-clustering results applied the Partitioning Around Medoids (PAM) algorithm and Multidimensional Scaling (MDS), and are consistent with the results from SAS clustering. In spite of extensive efforts to provide user information with recommendation systems, most projects are successful only when companies have sufficient data about users and transactions. Our proposed methodology, user-perspective issue clustering, can provide practical support to decision-making in companies because it enhances user-related data from unstructured textual data. To overcome the problem of insufficient data from traditional approaches, our methodology infers customers' real interests by utilizing web transaction logs. In addition, we suggest topic analysis and issue clustering as a practical means of issue identification.

Design of Summer Very Short-term Precipitation Forecasting Pattern in Metropolitan Area Using Optimized RBFNNs (최적화된 다항식 방사형 기저함수 신경회로망을 이용한 수도권 여름철 초단기 강수예측 패턴 설계)

  • Kim, Hyun-Ki;Choi, Woo-Yong;Oh, Sung-Kwun
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.23 no.6
    • /
    • pp.533-538
    • /
    • 2013
  • The damage caused by Recent frequently occurring locality torrential rains is increasing rapidly. In case of densely populated metropolitan area, casualties and property damage is a serious due to landslides and debris flows and floods. Therefore, the importance of predictions about the torrential is increasing. Precipitation characteristic of the bad weather in Korea is divided into typhoons and torrential rains. This seems to vary depending on the duration and area. Rainfall is difficult to predict because regional precipitation is large volatility and nonlinear. In this paper, Very short-term precipitation forecasting pattern model is implemented using KLAPS data used by Korea Meteorological Administration. we designed very short term precipitation forecasting pattern model using GA-based RBFNNs. the structural and parametric values such as the number of Inputs, polynomial type,number of fcm cluster, and fuzzification coefficient are optimized by GA optimization algorithm.

The Habitat Classification of mammals in Korea based on the National Ecosystem Survey (전국자연환경조사를 활용한 포유류 서식지 유형의 분류)

  • Lee, Hwajin;Ha, Jeongwook;Cha, Jinyeol;Lee, Junghyo;Yoon, Heenam;Chung, Chulun;Oh, Hongshik;Bae, Soyeon
    • Journal of Environmental Impact Assessment
    • /
    • v.26 no.2
    • /
    • pp.160-170
    • /
    • 2017
  • The purpose of this study is to perform clustering of the habitat types and to identify the characteristics of species in the habitat types using mammal data (70,562) of the 3rd National Ecosystem Survey conducted from 2006 to 2012. The 15 habitat types recorded in the field-paper of the 3rd National ecosystem survey were reclassified, which was followed by the statistical analysis of mammal habitat types. In the habitat types cluster analysis, non-hierarchical cluster analysis (k-means cluster analysis), hierarchical cluster analysis, and non-metric multidimensional scaling method were applied to 14 habitat types recorded more than 30 times. A total of 7 Orders, 16 Families, and 39 Species of mammals were identified in the 3rd National Ecosystem Survey collected nationwide. When 11 clusters were classified by habitat types, the simple structure index was the highest (ssi = 0.07). As a result of the similarities and hierarchies between habitat types suggested by the hierarchical clustering analysis, the residential areas were the most different habitat types for mammals; the next following type was a cluster together with rivers and coasts. The results of the non-metric multidimensional scaling analysis demonstrated that both Mus musculus and Rattus norvegicus restrictively appeared in a residential area, which is the most discriminating habitat type. Lutra lutra restrictively appeared in coastal and river areas. In summary, according to our results, the mammalian habitat can be divided into the following four types: (1) the forest type (using forest as the main habitat and migration route); (2) the river type (using water as the main habitat); (3) the residence habitat (living near residential area); and (4) the lowland type (consuming grain or seeds as the main feeding resource).

Modeling of the Cluster-based Multi-hop Sensor Networks (클거스터 기반 다중 홉 센서 네트워크의 모델링 기법)

  • Choi Jin-Chul;Lee Chae-Woo
    • Journal of the Institute of Electronics Engineers of Korea TC
    • /
    • v.43 no.1 s.343
    • /
    • pp.57-70
    • /
    • 2006
  • This paper descWireless Sensor Network consisting of a number of small sensors with transceiver and data processor is an effective means for gathering data in a variety of environments. The data collected by each sensor is transmitted to a processing center that use all reported data to estimate characteristics of the environment or detect an event. This process must be designed to conserve the limited energy resources of the sensor since neighboring sensors generally have the data of similar information. Therefore, clustering scheme which sends aggregated information to the processing center may save energy. Existing multi-hop cluster energy consumption modeling scheme can not estimate exact energy consumption of an individual sensor. In this paper, we propose a new cluster energy consumption model which modified existing problem. We can estimate more accurate total energy consumption according to the number of clusterheads by using Voronoi tessellation. Thus, we can realize an energy efficient cluster formation. Our modeling has an accuracy over $90\%$ when compared with simulation and has considerably superior than existing modeling scheme about $60\%.$ We also confirmed that energy consumption of the proposed modeling scheme is more accurate when the sensor density is increased.

Development of Customer Sentiment Pattern Map for Webtoon Content Recommendation (웹툰 콘텐츠 추천을 위한 소비자 감성 패턴 맵 개발)

  • Lee, Junsik;Park, Do-Hyung
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.4
    • /
    • pp.67-88
    • /
    • 2019
  • Webtoon is a Korean-style digital comics platform that distributes comics content produced using the characteristic elements of the Internet in a form that can be consumed online. With the recent rapid growth of the webtoon industry and the exponential increase in the supply of webtoon content, the need for effective webtoon content recommendation measures is growing. Webtoons are digital content products that combine pictorial, literary and digital elements. Therefore, webtoons stimulate consumer sentiment by making readers have fun and engaging and empathizing with the situations in which webtoons are produced. In this context, it can be expected that the sentiment that webtoons evoke to consumers will serve as an important criterion for consumers' choice of webtoons. However, there is a lack of research to improve webtoons' recommendation performance by utilizing consumer sentiment. This study is aimed at developing consumer sentiment pattern maps that can support effective recommendations of webtoon content, focusing on consumer sentiments that have not been fully discussed previously. Metadata and consumer sentiments data were collected for 200 works serviced on the Korean webtoon platform 'Naver Webtoon' to conduct this study. 488 sentiment terms were collected for 127 works, excluding those that did not meet the purpose of the analysis. Next, similar or duplicate terms were combined or abstracted in accordance with the bottom-up approach. As a result, we have built webtoons specialized sentiment-index, which are reduced to a total of 63 emotive adjectives. By performing exploratory factor analysis on the constructed sentiment-index, we have derived three important dimensions for classifying webtoon types. The exploratory factor analysis was performed through the Principal Component Analysis (PCA) using varimax factor rotation. The three dimensions were named 'Immersion', 'Touch' and 'Irritant' respectively. Based on this, K-Means clustering was performed and the entire webtoons were classified into four types. Each type was named 'Snack', 'Drama', 'Irritant', and 'Romance'. For each type of webtoon, we wrote webtoon-sentiment 2-Mode network graphs and looked at the characteristics of the sentiment pattern appearing for each type. In addition, through profiling analysis, we were able to derive meaningful strategic implications for each type of webtoon. First, The 'Snack' cluster is a collection of webtoons that are fast-paced and highly entertaining. Many consumers are interested in these webtoons, but they don't rate them well. Also, consumers mostly use simple expressions of sentiment when talking about these webtoons. Webtoons belonging to 'Snack' are expected to appeal to modern people who want to consume content easily and quickly during short travel time, such as commuting time. Secondly, webtoons belonging to 'Drama' are expected to evoke realistic and everyday sentiments rather than exaggerated and light comic ones. When consumers talk about webtoons belonging to a 'Drama' cluster in online, they are found to express a variety of sentiments. It is appropriate to establish an OSMU(One source multi-use) strategy to extend these webtoons to other content such as movies and TV series. Third, the sentiment pattern map of 'Irritant' shows the sentiments that discourage customer interest by stimulating discomfort. Webtoons that evoke these sentiments are hard to get public attention. Artists should pay attention to these sentiments that cause inconvenience to consumers in creating webtoons. Finally, Webtoons belonging to 'Romance' do not evoke a variety of consumer sentiments, but they are interpreted as touching consumers. They are expected to be consumed as 'healing content' targeted at consumers with high levels of stress or mental fatigue in their lives. The results of this study are meaningful in that it identifies the applicability of consumer sentiment in the areas of recommendation and classification of webtoons, and provides guidelines to help members of webtoons' ecosystem better understand consumers and formulate strategies.

A Study on the Impact Factors of Contents Diffusion in Youtube using Integrated Content Network Analysis (일반영향요인과 댓글기반 콘텐츠 네트워크 분석을 통합한 유튜브(Youtube)상의 콘텐츠 확산 영향요인 연구)

  • Park, Byung Eun;Lim, Gyoo Gun
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.3
    • /
    • pp.19-36
    • /
    • 2015
  • Social media is an emerging issue in content services and in current business environment. YouTube is the most representative social media service in the world. YouTube is different from other conventional content services in its open user participation and contents creation methods. To promote a content in YouTube, it is important to understand the diffusion phenomena of contents and the network structural characteristics. Most previous studies analyzed impact factors of contents diffusion from the view point of general behavioral factors. Currently some researchers use network structure factors. However, these two approaches have been used separately. However this study tries to analyze the general impact factors on the view count and content based network structures all together. In addition, when building a content based network, this study forms the network structure by analyzing user comments on 22,370 contents of YouTube not based on the individual user based network. From this study, we re-proved statistically the causal relations between view count and not only general factors but also network factors. Moreover by analyzing this integrated research model, we found that these factors affect the view count of YouTube according to the following order; Uploader Followers, Video Age, Betweenness Centrality, Comments, Closeness Centrality, Clustering Coefficient and Rating. However Degree Centrality and Eigenvector Centrality affect the view count negatively. From this research some strategic points for the utilizing of contents diffusion are as followings. First, it is needed to manage general factors such as the number of uploader followers or subscribers, the video age, the number of comments, average rating points, and etc. The impact of average rating points is not so much important as we thought before. However, it is needed to increase the number of uploader followers strategically and sustain the contents in the service as long as possible. Second, we need to pay attention to the impacts of betweenness centrality and closeness centrality among other network factors. Users seems to search the related subject or similar contents after watching a content. It is needed to shorten the distance between other popular contents in the service. Namely, this study showed that it is beneficial for increasing view counts by decreasing the number of search attempts and increasing similarity with many other contents. This is consistent with the result of the clustering coefficient impact analysis. Third, it is important to notice the negative impact of degree centrality and eigenvector centrality on the view count. If the number of connections with other contents is too much increased it means there are many similar contents and eventually it might distribute the view counts. Moreover, too high eigenvector centrality means that there are connections with popular contents around the content, and it might lose the view count because of the impact of the popular contents. It would be better to avoid connections with too powerful popular contents. From this study we analyzed the phenomenon and verified diffusion factors of Youtube contents by using an integrated model consisting of general factors and network structure factors. From the viewpoints of social contribution, this study might provide useful information to music or movie industry or other contents vendors for their effective contents services. This research provides basic schemes that can be applied strategically in online contents marketing. One of the limitations of this study is that this study formed a contents based network for the network structure analysis. It might be an indirect method to see the content network structure. We can use more various methods to establish direct content network. Further researches include more detailed researches like an analysis according to the types of contents or domains or characteristics of the contents or users, and etc.