• Title/Summary/Keyword: word clustering

Search Result 190, Processing Time 0.035 seconds

Implementation of a Web Document Clustering System Using Word2Vec (Word2Vec을 이용한 웹 문서 클러스터링 시스템 구현)

  • Yi, Hyun Seok;Ahn, Sung Hun;Lee, Yong Hwan;Cheon, Myung Jae;Park, Hyeok Ju;Park, Mee Hwa;Lee, Yong Kyu
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2016.10a
    • /
    • pp.26-29
    • /
    • 2016
  • 웹 문서 추천 시스템에서는 유사한 내용의 문서임에도 불구하고 URL이 달라서 다른 문서로 인식하여 사용자에게 추천하는 데이터 희소성 문제가 있다. 여기서 기존 연구들은 이 문제에 대한 해결 방법으로 TF-IDF를 이용하였으나 비용 및 시간의 한계가 있으며 유의어 분류 문제가 있다. 본 논문에서는 Word2Vec을 이용한 웹문서 학습 시스템을 통해 문제를 해결한다. 제안 시스템은 언론사의 뉴스를 수집하고 이를 정형화된 형식으로 분석하여 가공하는 전처리 과정을 거친 후 Word2Vec 학습을 통해 문서 벡터를 생성하고 이를 K-Means 클러스터링으로 유사 문서군으로 분류한다. 이 시스템을 이용하면 데이터 희소성 문제를 해결할 뿐만 아니라 연산량이 TF-IDF에 비해 줄어들고 유의어 분류 시 유사도가 높아지는 강점이 있다.

The Effectiveness of Hierarchic Clustering on Query Results in OPAC (OPAC에서 탐색결과의 클러스터링에 관한 연구)

  • Ro, Jung-Soon
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.38 no.1
    • /
    • pp.35-50
    • /
    • 2004
  • This study evaluated the applicability of the static hierarchic clustering model to clustering query results in OPAC. Two clustering methods(Between Average Linkage(BAL) and Complete Linkage(CL)) and two similarity coefficients(Dice and Jaccard) were tested on the query results retrieved from 16 title-based keyword searchings. The precision of optimal dusters was improved more than 100% compared with title-word searching. There was no difference between similarity coefficients but clustering methods in optimal cluster effectiveness. CL method is better in precision ratio but BAL is better in recall ratio at the optimal top-level and bottom-level clusters. However the differences are not significant except higher recall ratio of BAL at the top-level duster. Small number of clusters and long chain of hierarchy for optimal cluster resulted from BAL could not be desirable and efficient.

Determining on Model-based Clusters of Time Series Data (시계열데이터의 모델기반 클러스터 결정)

  • Jeon, Jin-Ho;Lee, Gye-Sung
    • The Journal of the Korea Contents Association
    • /
    • v.7 no.6
    • /
    • pp.22-30
    • /
    • 2007
  • Most real word systems such as world economy, stock market, and medical applications, contain a series of dynamic and complex phenomena. One of common methods to understand these systems is to build a model and analyze the behavior of the system. In this paper, we investigated methods for best clustering over time series data. As a first step for clustering, BIC (Bayesian Information Criterion) approximation is used to determine the number of clusters. A search technique to improve clustering efficiency is also suggested by analyzing the relationship between data size and BIC values. For clustering, two methods, model-based and similarity based methods, are analyzed and compared. A number of experiments have been performed to check its validity using real data(stock price). BIC approximation measure has been confirmed that it suggests best number of clusters through experiments provided that the number of data is relatively large. It is also confirmed that the model-based clustering produces more reliable clustering than similarity based ones.

A Comparative Study on Clustering Methods for Grouping Related Tags (연관 태그의 군집화를 위한 클러스터링 기법 비교 연구)

  • Han, Seung-Hee
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.43 no.3
    • /
    • pp.399-416
    • /
    • 2009
  • In this study, clustering methods with related tags were discussed for improving search and exploration in the tag space. The experiments were performed on 10 Delicious tags and the strongly-related tags extracted by each 300 documents, and hierarchical and non-hierarchical clustering methods were carried out based on the tag co-occurrences. To evaluate the experimental results, cluster relevance was measured. Results showed that Ward's method with cosine coefficient, which shows good performance to term clustering, was best performed with consistent clustering tendency. Furthermore, it was analyzed that cluster membership among related tags is based on users' tagging purposes or interest and can disambiguate word sense. Therefore, tag clusters would be helpful for improving search and exploration in the tag space.

Examining the Intellectual Structure of Reading Studies with Co-Word Analysis Based on the Importance of Journals and Sequence of Keywords (학술지 중요도와 키워드 순서를 고려한 단어동시출현 분석을 이용한 독서분야의 지적구조 분석)

  • Zhang, Ling Ling;Hong, Hyun Jin
    • Journal of the Korean BIBLIA Society for library and Information Science
    • /
    • v.25 no.1
    • /
    • pp.295-318
    • /
    • 2014
  • The purpose of this study is to analyze the intellectual structure of reading studies by using Co-Word Analysis based on the mixed weight in which the level of academic journals and the position of keywords are calculated. To achieve it, 838 academic articles relating to reading studies from KCI during the period from 2003 to 2012 were retrieved and 56 keywords were extracted. The results of clustering analysis, MDS, network analysis are that the network based on the mixed weight has a better performance in above three methods and reading studies can be divided into 4 bigger divisions and 11 subdivisions. Finally, the result of document analysis shows reading studies changes its research tendency from theoretical studies to empirical studies.

A Study on the Intellectual Structure of Data Science Using Co-Word Analysis (동시출현단어분석을 통한 데이터과학 분야의 지적구조에 관한 연구)

  • Kim, Hyunjung
    • Journal of the Korean Society for information Management
    • /
    • v.34 no.4
    • /
    • pp.101-126
    • /
    • 2017
  • Data Science is emerging as a closely related field of study to Library and Information Science (LIS), and as an interdisciplinary subject combining LIS, statistics and computer science in an attempt to understand the value of data by applying what LIS has been doing for collecting, storing, organizing, analyzing, and utilizing information. To investigate which subject fields other than LIS, statistics, and computer science are related to Data Science, this study retrieved 667 materials from Web of Science Core Collection, extracted terms representing Web of Science Categories, examined subject fields that are studying Data Science using descriptive analysis, analyzed the intellectual structure of the field by co-word analysis and network analysis, and visualized the results as a Pathfinder network with clustering created with the PNNC clustering algorithm. The result of this study might help to understand the intellectual structure of the Data Science field, and may be helpful to give an idea for developing relatively new curriculum.

Design and Implementation of Topic Map Generation System based Tag (태그 기반 토픽맵 생성 시스템의 설계 및 구현)

  • Lee, Si-Hwa;Lee, Man-Hyoung;Hwang, Dae-Hoon
    • Journal of Korea Multimedia Society
    • /
    • v.13 no.5
    • /
    • pp.730-739
    • /
    • 2010
  • One of core technology in Web 2.0 is tagging, which is applied to multimedia data such as web document of blog, image and video etc widely. But unlike expectation that the tags will be reused in information retrieval and then maximize the retrieval efficiency, unacceptable retrieval results appear owing to toot limitation of tag. In this paper, in the base of preceding research about image retrieval through tag clustering, we design and implement a topic map generation system which is a semantic knowledge system. Finally, tag information in cluster were generated automatically with topics of topic map. The generated topics of topic map are endowed with mean relationship by use of WordNet. Also the topics are endowed with occurrence information suitable for topic pair, and then a topic map with semantic knowledge system can be generated. As the result, the topic map preposed in this paper can be used in not only user's information retrieval demand with semantic navigation but alse convenient and abundant information service.

Weighted Bayesian Automatic Document Categorization Based on Association Word Knowledge Base by Apriori Algorithm (Apriori알고리즘에 의한 연관 단어 지식 베이스에 기반한 가중치가 부여된 베이지만 자동 문서 분류)

  • 고수정;이정현
    • Journal of Korea Multimedia Society
    • /
    • v.4 no.2
    • /
    • pp.171-181
    • /
    • 2001
  • The previous Bayesian document categorization method has problems that it requires a lot of time and effort in word clustering and it hardly reflects the semantic information between words. In this paper, we propose a weighted Bayesian document categorizing method based on association word knowledge base acquired by mining technique. The proposed method constructs weighted association word knowledge base using documents in training set. Then, classifier using Bayesian probability categorizes documents based on the constructed association word knowledge base. In order to evaluate performance of the proposed method, we compare our experimental results with those of weighted Bayesian document categorizing method using vocabulary dictionary by mutual information, weighted Bayesian document categorizing method, and simple Bayesian document categorizing method. The experimental result shows that weighted Bayesian categorizing method using association word knowledge base has improved performance 0.87% and 2.77% and 5.09% over weighted Bayesian categorizing method using vocabulary dictionary by mutual information and weighted Bayesian method and simple Bayesian method, respectively.

  • PDF

Incidence of Online Public Opinion on Guangzhou Simultaneous Renting and Purchasing Policy - A data mining application

  • Wang, Yancheng;Li, Haixian
    • Asian Journal for Public Opinion Research
    • /
    • v.5 no.4
    • /
    • pp.266-284
    • /
    • 2018
  • This paper adopts the big data research method, and draws 491 data from the Tianya Forum about the Simultaneous Renting and Purchasing policy of Guangzhou. The qualitative analysis software Nvivo11 is used to cluster the main questions about the Simultaneous Renting and Purchasing policy in the forum. The 36 high-frequency word frequencies are obtained through text clustering. Through rooted theory analysis, the main driving factors for summarizing people's doubts are 9 main categories, 3 core categories, and the model of driving factors for online forums is established. The study finds that resource factors are the most key factor, economic factors are the important drivers, and policy guiding factors are sub-important drivers.

Identifying the Diffusion Patterns of Movies by Opening Strength and Profitability (개봉 규모와 수익성에 따른 영화의 분류와 확산 패턴 분석)

  • Kim, Taegu;Hong, Jungsik
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.39 no.5
    • /
    • pp.412-421
    • /
    • 2013
  • Motion picture industry is one of the most representative fields in the cultural industry and has experienced constant growth both worldwide and within domestic markets. However, little research has been undertaken for diffusion patterns of motion pictures, whereas various issues such as demand forecasting and success factor analysis have been widely explored. To analyze diffusion patterns, we adopted extended Bass model to reflect the potential demand of movies. Four clusters of selected movies were derived by k-means clustering method with criteria of opening strength and profitability and then compared by their diffusion patterns. Results indicated that movies with high profitability and medium opening strength are most significantly influenced by word of mouth effect, while low profitability movies display nearly monotonic decreasing diffusion patterns with noticeable initial adoption rates and relatively early peak points in their runs.