• Journal of Information Science Theory and Practice
• /
• v.8 no.2
• /
• pp.6-17
• /
• 2020

Word similarity is often measured to enhance system performance in the information retrieval field and other related areas. This paper reports on an experimental comparison of values for word similarity measures that were computed based on 50 intentionally selected words from a Reuters corpus. There were three targets, including (1) co-occurrence-based similarity measures (for which a co-occurrence frequency is counted as the number of documents or sentences), (2) context-based distributional similarity measures obtained from a latent Dirichlet allocation (LDA), nonnegative matrix factorization (NMF), and Word2Vec algorithm, and (3) similarity measures computed from the tf-idf weights of each word according to a vector space model (VSM). Here, a Pearson correlation coefficient for a pair of VSM-based similarity measures and co-occurrence-based similarity measures according to the number of documents was highest. Group-average agglomerative hierarchical clustering was also applied to similarity matrices computed by individual measures. An evaluation of the cluster sets according to an answer set revealed that VSM- and LDA-based similarity measures performed best.

• Journal of the Korean Society for information Management
• /
• v.24 no.1 s.63
• /
• pp.103-120
• /
• 2007

Node is the semantic unit and a part of structured document. Information retrieval from structured documents offers an opportunity to go subdivided below the document level in search of relevant information, making any element in an structured document a retrievable unit. The node-based document retrieval constitutes several similarity calculating methods and the extended node retrieval method using structure information. Retrieval performance is hardly influenced by the methods for determining document similarity The extended node method outperformed the others as a whole.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.13 no.5
• /
• pp.37-47
• /
• 2013

Text Mining is a research area of retrieving high quality hidden information such as patterns, trends, or distributions through analyzing unformatted text. Basically, since text mining assumes an unstructured text, it needs to be represented as a simple text model for analyzing it. So far, most frequently used model is VSM(Vector Space Model), in which a text is represented as a bag of words. However, recently much researches tried to apply a graph-based text model for representing semantic relationships between words. In this paper, we survey research trends of graph-based text representation models for text mining. Additionally, we also discuss about future models of graph-based text mining.

• Journal of Korean Society of Rural Planning
• /
• v.23 no.2
• /
• pp.29-42
• /
• 2017

Sustainable development is no longer an option, but a requirement. Under this awareness, UN adopted 17 goals for a new sustainable development agenda on September 2015, named 'Sustainable Development Goals(SDGs)'. The Korean Society of Rural Planning(KSRP) is established on July 1994 for the sustainable development of rural areas. On the purpose to quantitatively analyze the research trend of KSRP's publications with the viewpoint of SDGs, the qualitative documents of 17 SDGs and 771 publications were mathematically transformed into vectors and the similarity was numerically measured with the 'Vector Space Model(VSM)'. The results show that 'Sustainable cities and communities(SDG 11)', 'Zero hunger(SDG 2)', 'Life on land(SDG 15)' and 'Responsible consumption and production(SDG 12)' have strong relationships with KSRP, while those of 'Affordable and clean energy(SDG 7)', 'Peace, justice and strong institution(SDG 16)' and 'Gender equality(SDG 5)' are weak. It is also found that the relationships of KSRP publications with 'energy' and 'climate change' issues(SDG 7, 13) were greatly increased during the period of 1995-2016, in spite of their weak relationships.

• The Journal of the Korea Contents Association
• /
• v.9 no.3
• /
• pp.1-8
• /
• 2009

This paper proposes a new document clustering system using fuzzy logic-based genetic algorithm (GA) and semantic vector expansion technology. It has been known in many GA papers that the success depends on two factors, the diversity of the population and the capability to convergence. We use the fuzzy logic-based operators to adaptively adjust the influence between these two factors. In traditional document clustering, the most popular and straightforward approach to represent the document is vector space model (VSM). However, this approach not only leads to a high dimensional feature space, but also ignores the semantic relationships between some important words, which would affect the accuracy of clustering. In this paper we use latent semantic analysis (LSA)to expand the documents to corresponding semantic vectors conceptually, rather than the individual terms. Meanwhile, the sizes of the vectors can be reduced drastically. We test our clustering algorithm on 20 news groups and Reuter collection data sets. The results show that our method outperforms the conventional GA in various document representation environments.

• Journal of KIISE
• /
• v.41 no.12
• /
• pp.1117-1125
• /
• 2014

Folksonomy data, which is derived from social tagging systems, is a useful source for understanding a user's intention and interest. Using the folksonomy data, it is possible to create an accurate user profile which can be utilized to build a personalized search system. However there are limitations in some of the traditional methods such as Vector Space Model(VSM) for user profiling and similarity computation. This paper suggests a novel method with graph-based user and document profile which uses the proximity information of query terms to improve personalized search. We demonstrate the performance of the suggested method by comparing its performance with several state-of-the-art VSM based personalization models in two different folksonomy datasets. The results show that the proposed model constantly outperforms the other state-of-the-art personalization models. Furthermore, the parameter sensitivity results show that the proposed model is parameter-free in that it is not affected by the idiosyncratic nature of datasets.

• Proceedings of the Korean Information Science Society Conference
• /
• 2003.04c
• /
• pp.440-442
• /
• 2003

문서 집합 내의 개념 또는 의미 관계의 자동 분석은 보다 효율적인 정보 획득과 단어수준 이상의 개념 수준에서의 운서 비교를 가능하게 한다. 본 논문에서는 은닉변수모델을 이용하여 문서 집합으로부터 단어들 간의 의미관계를 자동적으로 추출하고 이를 통해 문서간 유사도 측정을 효과적으로 하기 위한 방안을 제시한다. 은닉변수 모델로는 다중요인모델의 학습이 용이한 헬름홀츠 머신을 활용하묘 이의 학습 결과에 기반하여, 문서간 비교를 한 의미 커널(semantic kernel)을 구축한다. 2개의 문서 집합 HEDLINE과 CACM 데이터에 대한 검색 실험에서, 제안된 기법을 적응함으로써 기본 VSM(Vector Space Model) 에 비해 20% 이상의 평균 정확도 향상을 이를 수 있었다.

• Proceedings of the Korean Operations and Management Science Society Conference
• /
• 2006.05a
• /
• pp.1198-1206
• /
• 2006

프로그램 소스코드 표절 검사에 대한 기존 방법은 크게 지문(finger-print)법과 구조기반 검사법으로 나뉘며, 주로 단어의 유사성이나 발생빈도를 사용하거나 소스코드 구조상의 특징으로 두 소스간의 유사성을 비교한다. 본 연구에서는 프로그래밍 언어의 예약어 시퀀스를 사용하여 소스코드들 간의 유사성을 비교하고, 이 결과를 FCA(Formal Concept Analysis)를 통해 해석하고 시각화 하는 방법을 제시한다. 일반적인 VSM(Vector Space Model)과 같은 단일 단어 분석으로는 단어의 인접성을 구분할 수 없으므로 단어의 시퀀스 분석이 가능하도록 알고리즘을 구성하였으며 이러한 방식은 지문법의 단점인 소스코드의 부분적인 표절 탐지의 난점을 해결할 수 있고 함수의 호출 순서나 수행 순서에 상관없이 표절을 탐지할 수 있는 장점을 가진다. 마지막으로 유사도 측정결과는 FCA를 이용하여 격자(lattice)로 시각화됨으로써 이용자의 이해도를 높일 수 있다.

• The KIPS Transactions:PartA
• /
• v.12A no.6 s.96
• /
• pp.539-546
• /
• 2005

Recently, as the number of Internet users are growing explosively, e-learning has been applied spread, as well as remote evaluation of intellectual capacity However, only the multiple choice and/or the objective tests have been applied to the e-learning, because of difficulty of natural language processing. For the intelligent marking of short-essay typed answer papers with rapidness and fairness, this work utilize heterogenous linguistic knowledges. Firstly, we construct the semantic kernel from un tagged corpus. Then the answer papers of students and instructors are transformed into the vector form. Finally, we evaluate the similarity between the papers by using the semantic kernel and decide whether the answer paper is correct or not, based on the similarity values. For the construction of the semantic kernel, we used latent semantic analysis based on the vector space model. Further we try to reduce the problem of information shortage, by integrating Korean Word Net. For the construction of the semantic kernel we collected 38,727 newspaper articles and extracted 75,175 indexed terms. In the experiment, about 0.894 correlation coefficient value, between the marking results from this system and the human instructors, was acquired.