• Title/Summary/Keyword: Lexical similarity

Search Result 39, Processing Time 0.027 seconds

Semantic Similarity Measures Between Words within a Document using WordNet (워드넷을 이용한 문서내에서 단어 사이의 의미적 유사도 측정)

  • Kang, SeokHoon;Park, JongMin
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.16 no.11
    • /
    • pp.7718-7728
    • /
    • 2015
  • Semantic similarity between words can be applied in many fields including computational linguistics, artificial intelligence, and information retrieval. In this paper, we present weighted method for measuring a semantic similarity between words in a document. This method uses edge distance and depth of WordNet. The method calculates a semantic similarity between words on the basis of document information. Document information uses word term frequencies(TF) and word concept frequencies(CF). Each word weight value is calculated by TF and CF in the document. The method includes the edge distance between words, the depth of subsumer, and the word weight in the document. We compared out scheme with the other method by experiments. As the result, the proposed method outperforms other similarity measures. In the document, the word weight value is calculated by the proposed method. Other methods which based simple shortest distance or depth had difficult to represent the information or merge informations. This paper considered shortest distance, depth and information of words in the document, and also improved the performance.

A New Similarity Measure for e-Catalog Retrieval Based on Semantic Relationship (의미적 연결 관계에 기반한 전자 카탈로그 검색용 유사도 척도)

  • Seo, Kwang-Hun;Lee, Sang-Goo
    • Journal of KIISE:Databases
    • /
    • v.34 no.6
    • /
    • pp.554-563
    • /
    • 2007
  • The e-Marketplace is growing rapidly and providing a more complex relationship between providers and consumers. In recent years, e-Marketplace integration or cooperation issues have become an important issue in e-Business. The e-Catalog is a key factor in e-Business, which means an e-Catalog System needs to contain more large data and requires a more efficient retrieval system. This paper focuses on designing an efficient retrieval system for very large e-Catalogs of large e-Marketplaces. For this reason, a new similarity measure for e-Catalog retrieval based on semantic relationships was proposed. Our achievement is this: first, a new e-Catalog data model based on semantic relationships was designed. Second, the model was extended by considering lexical features (Especially, focus on Korean). Third, the factors affecting similarity with the model was defined. Fourth, from the factors, we finally defined a new similarity measure, realized the system and verified it through experimentation.

The Effects of Visual and Phonological Similarity on Hanja Word Recognition (시각 형태 정보와 소리 정보가 한자 단어 재인에 미치는 영향)

  • Nam, Ki-Chun
    • Annual Conference on Human and Language Technology
    • /
    • 1995.10a
    • /
    • pp.244-252
    • /
    • 1995
  • 본 연구는 한자를 이용하여 시각 정보 (Visual Information)와 음성 정보(Phonological Information)가 단어 재인과 단어 명명 과정에 어떻게 영향을 주는 지를 조사하기 위하여 실시되었다. 기존의 영어를 이용한 연구에서는 시각 정보와 음성 정보를 독립적으로 조작할 수 없었기에 두 요소가 단어 재인에 어떤 영향을 주는 지를 살피는데 어려움이 있었다. 그러나 한자단어를 이용하면 시각 정보와 음성 정보를 독립적으로 조작할 수 있기 때문에 영어 단어를 사용하는 것보다 유리하다. 본 실험에서는 한자 단어를 이용하여 점화 단어 (Prime Word)와 목표 단어(Target Word)간의 시간간격(SOA)을 100 ms, 200 ms, 750 ms, 그리고 2000 ms로 변화시키면서 시간이 흐름에 따라 시각적 유사성과 음성적 유사성에 의한 점화 효과(Priming Effect)가 어떻게 변화하는 지를 조사하였다. 이 실험 결과에 의하면, 100 ms 조건에서는 시각적 유사성에 의한 점화 효과만 있었다. 그러나, 200 ms, 750 ms, 2000 ms 조건들에서는 시각적 유사성뿐만 아니라 음성적 유사성에 의해서도 점화효과가 있었다. 이와 같은 실험 결과는 최초의 한자 단어의 어휘 접근 (Lexical Access)이 시각 정보에 의해 결정됨을 보여주고 있다.

  • PDF

An XML Keyword Indexing Method Using on Lexical Similarity (단락을 분류에 따른 XML 키워드 가중치 결정 기법)

  • Jeong, Hye-Jin;Kim, Hyoung-Jin
    • Proceedings of the KAIS Fall Conference
    • /
    • 2008.05a
    • /
    • pp.205-208
    • /
    • 2008
  • 보다 효과적인 키워드 추출 및 키워드 가중치 결정을 위하여 문서의 내용뿐 아니라 구조를 이용하여 색인을 추출하는 연구가 이루어지고 있는데, 대부분의 연구들이 XML 단락별 중요도가 아닌, 문맥상의 단락에 대한 중요도를 계산하는게 일반적이다. 이러한 기존 연구들은 대부분이 객관적인 실험을 통해서 중요도를 입증하기보다는 일반적인 관점에서 단순한 수치로 중요도를 결정하고 있다. 본 논문에서는 웹 문서 관리를 위한 표준으로 자리잡아가고 있는 XML 문서의 자동색인을 위하여, 논문을 구성하는 주요 단락을 세분하고, 단락에서 추출된 용어의 가중치를 갱신해 가면서 최종 색인어 가중치를 계산하는 방법을 제안한다.

  • PDF

Citation-based Article Summarization using a Combination of Lexical Text Similarities: Evaluation with Computational Linguistics Literature Summarization Datasets

  • Kang, In-Su
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.7
    • /
    • pp.31-37
    • /
    • 2019
  • Citation-based article summarization is to create a shortened text for an academic article, reflecting the content of citing sentences which contain other's thoughts about the target article to be summarized. To deal with the problem, this study introduces an extractive summarization method based on calculating a linear combination of various sentence salience scores, which represent the degrees to which a candidate sentence reflects the content of author's abstract text, reader's citing text, and the target article to be summarized. In the current study, salience scores are obtained by computing surface-level textual similarities. Experiments using CL-SciSumm datasets show that the proposed method parallels or outperforms the previous approaches in ROUGE evaluations against SciSumm-2017 human summaries and SciSumm-2016/2017 community summaries.

Methodology of Developing Train Set for BERT's Sentence Similarity Classification with Lexical Mismatch (어휘 유사 문장 판별을 위한 BERT모델의 학습자료 구축)

  • Jeong, Jaehwan;Kim, Dongjun;Lee, Woochul;Lee, Yeonsoo
    • Annual Conference on Human and Language Technology
    • /
    • 2019.10a
    • /
    • pp.265-271
    • /
    • 2019
  • 본 논문은 어휘가 비슷한 문장들을 효과적으로 분류하는 BERT 기반 유사 문장 분류기의 학습 자료 구성 방법을 제안한다. 기존의 유사 문장 분류기는 문장의 의미와 상관 없이 각 문장에서 출현한 어휘의 유사도를 기준으로 분류하였다. 이는 학습 자료 내의 유사 문장 쌍들이 유사하지 않은 문장 쌍들보다 어휘 유사도가 높기 때문이다. 따라서, 본 논문은 어휘 유사도가 높은 유사 의미 문장 쌍들과 어휘 유사도가 높지 않은 의미 문장 쌍들을 학습 자료에 추가하여 BERT 유사 문장 분류기를 학습하여 전체 분류 성능을 크게 향상시켰다. 이는 문장의 의미를 결정짓는 단어들과 그렇지 않은 단어들을 유사 문장 분류기가 학습하였기 때문이다. 제안하는 학습 데이터 구축 방법을 기반으로 학습된 BERT 유사 문장 분류기들의 학습된 self-attention weight들을 비교 분석하여 BERT 내부에서 어떤 변화가 발생하였는지 확인하였다.

  • PDF

Ontology Selection Ranking Model based on Semantic Similarity Approach (의미적 유사성에 기반한 온톨로지 선택 랭킹 모델)

  • Oh, Sun-Ju;Ahn, Joong-Ho;Park, Jin-Soo
    • The Journal of Society for e-Business Studies
    • /
    • v.14 no.2
    • /
    • pp.95-116
    • /
    • 2009
  • Ontologies have provided supports in integrating heterogeneous and distributed information. More and more ontologies and tools have been developed in various domains. However, building ontologies requires much time and effort. Therefore, ontologies need to be shared and reused among users. Specifically, finding the desired ontology from an ontology repository will benefit users. In the past, most of the studies on retrieving and ranking ontologies have mainly focused on lexical level supports. In those cases, it is impossible to find an ontology that includes concepts that users want to use at the semantic level. Most ontology libraries and ontology search engines have not provided semantic matching capability. Retrieving an ontology that users want to use requires a new ontology selection and ranking mechanism based on semantic similarity matching. We propose an ontology selection and ranking model consisting of selection criteria and metrics which are enhanced in semantic matching capabilities. The model we propose presents two novel features different from the previous research models. First, it enhances the ontology selection and ranking method practically and effectively by enabling semantic matching of taxonomy or relational linkage between concepts. Second, it identifies what measures should be used to rank ontologies in the given context and what weight should be assigned to each selection measure.

  • PDF

One-Class Classification Model Based on Lexical Information and Syntactic Patterns (어휘 정보와 구문 패턴에 기반한 단일 클래스 분류 모델)

  • Lee, Hyeon-gu;Choi, Maengsik;Kim, Harksoo
    • Journal of KIISE
    • /
    • v.42 no.6
    • /
    • pp.817-822
    • /
    • 2015
  • Relation extraction is an important information extraction technique that can be widely used in areas such as question-answering and knowledge population. Previous studies on relation extraction have been based on supervised machine learning models that need a large amount of training data manually annotated with relation categories. Recently, to reduce the manual annotation efforts for constructing training data, distant supervision methods have been proposed. However, these methods suffer from a drawback: it is difficult to use these methods for collecting negative training data that are necessary for resolving classification problems. To overcome this drawback, we propose a one-class classification model that can be trained without using negative data. The proposed model determines whether an input data item is included in an inner category by using a similarity measure based on lexical information and syntactic patterns in a vector space. In the experiments conducted in this study, the proposed model showed higher performance (an F1-score of 0.6509 and an accuracy of 0.6833) than a representative one-class classification model, one-class SVM(Support Vector Machine).

A Schema Matching Algorithm for an Automated Transformation of XML Documents (XML문서의 자동변환을 위한 스키마 매칭 알고리즘)

  • Lee Jun-Seung;Lee Kyong-Ho
    • Journal of Korea Multimedia Society
    • /
    • v.7 no.9
    • /
    • pp.1195-1207
    • /
    • 2004
  • Schema matching is prerequisite to an automated transformation of XML documents. Because previous works about schema matching compute all semantically-possible matchings, they produce many-to-many matching relationships. Such imprecise matchings are inappropriate for an automated transformation of XML documents. This paper presents an efficient schema matching algorithm that computes precise one-to-one matchings between two schemas. The proposed algorithm consists of two steps: preliminary matching relationships between leaf nodes in the two schemas are computed and one-to-one matchings are finally extracted based on a proposed path similarity. Specifically, for a sophisticated schema matching, the proposed algorithm is based on a domain ontology as well as a lexical database that includes abbreviations and synonyms. Experimental results with real schemas from an e-commerce field show that the proposed method is superior to previous works, resulting in an accuracy of 97% in average.

  • PDF

Graph-Based Word Sense Disambiguation Using Iterative Approach (반복적 기법을 사용한 그래프 기반 단어 모호성 해소)

  • Kang, Sangwoo
    • The Journal of Korean Institute of Next Generation Computing
    • /
    • v.13 no.2
    • /
    • pp.102-110
    • /
    • 2017
  • Current word sense disambiguation techniques employ various machine learning-based methods. Various approaches have been proposed to address this problem, including the knowledge base approach. This approach defines the sense of an ambiguous word in accordance with knowledge base information with no training corpus. In unsupervised learning techniques that use a knowledge base approach, graph-based and similarity-based methods have been the main research areas. The graph-based method has the advantage of constructing a semantic graph that delineates all paths between different senses that an ambiguous word may have. However, unnecessary semantic paths may be introduced, thereby increasing the risk of errors. To solve this problem and construct a fine-grained graph, in this paper, we propose a model that iteratively constructs the graph while eliminating unnecessary nodes and edges, i.e., senses and semantic paths. The hybrid similarity estimation model was applied to estimate a more accurate sense in the constructed semantic graph. Because the proposed model uses BabelNet, a multilingual lexical knowledge base, the model is not limited to a specific language.