• 제목/요약/키워드: Text semantic similarity

검색결과 49건 처리시간 0.026초

SRR을 이용한 분산 도메인 문서 객체 관리 (A Distributed Domain Document Object Management using Semantic Reference Relationship)

  • 이종득
    • 디지털융복합연구
    • /
    • 제10권5호
    • /
    • pp.267-273
    • /
    • 2012
  • 시맨틱 관계성은 포맷되지 않은 많은 문서 객체들을 계층적으로 구조화한다. 그러나 분산 응용도메인에서 관련 데이터를 추출하여 구조화하기란 쉽지 않는 일이다. 이러한 문제를 해결하기 위하여 본 논문에서는 분산된 응용 도메인 객체들을 서비스할 수 있도록 시멘틱 참조 관련성을 이용한 새로운 객체 관리 기법을 제안하였다. 제안된 기법은 응용 도메인 객체들로부터 시멘틱 유사성을 추출하기 위하여 프로파일 구조를 이용하였으며, 추출된 객체들의 시멘틱 관계성을 결정하기 위하여 joint matrix를 이용하였다. 제안된 기법의 성능을 알아보기 위하여 시뮬레이션을 수행하였으며, 시뮬레이션 결과 제안된 기법이 기존의 텍스트 마이닝 기법과 정보추출기법에 비해서 검색 성능이 우수함을 알게 되었다.

Ranking Translation Word Selection Using a Bilingual Dictionary and WordNet

  • Kim, Kweon-Yang;Park, Se-Young
    • 한국지능시스템학회논문지
    • /
    • 제16권1호
    • /
    • pp.124-129
    • /
    • 2006
  • This parer presents a method of ranking translation word selection for Korean verbs based on lexical knowledge contained in a bilingual Korean-English dictionary and WordNet that are easily obtainable knowledge resources. We focus on deciding which translation of the target word is the most appropriate using the measure of semantic relatedness through the 45 extended relations between possible translations of target word and some indicative clue words that play a role of predicate-arguments in source language text. In order to reduce the weight of application of possibly unwanted senses, we rank the possible word senses for each translation word by measuring semantic similarity between the translation word and its near synonyms. We report an average accuracy of $51\%$ with ten Korean ambiguous verbs. The evaluation suggests that our approach outperforms the default baseline performance and previous works.

워드 임베딩 기반 근사 Top-k 레이블 서브그래프 매칭 기법 (Approximate Top-k Labeled Subgraph Matching Scheme Based on Word Embedding)

  • 최도진;오영호;복경수;유재수
    • 한국콘텐츠학회논문지
    • /
    • 제22권8호
    • /
    • pp.33-43
    • /
    • 2022
  • 지식 그래프 및 단백질 상호 작용과 같은 실제 데이터에서 개체들과 개체들의 관계 및 구조를 나타내기 위해 레이블 그래프를 사용한다. IT의 급속한 발전과 데이터의 폭발적인 증가로 사용자에게 관심 있는 정보를 제공하기 위한 서브 그래프 매칭 기술이 필요하다. 본 논문은 레이블의 의미적 유사성과 그래프 구조 차이를 고려한 근사 Top-k 서브 그래프 매칭 기법을 제안한다. 제안하는 기법은 레이블 의미적 유사도를 고려하기 위하여 FastText을 활용한 학습 모델을 이용한다. 레이블 간 의미적 유사도를 미리 계산한 LSG(Label Similarity Graph)를 통해 처리 속도의 효율을 높인다. LSG를 통해 레이블이 정확하게 일치해야 확장이 가능한 기존 연구의 한계를 해결한다. 2-hop까지 탐색을 수행함으로써 질의 그래프에 대한 구조적 유사성을 지원한다. 매칭된 서브 그래프는 유사도 값 기반으로 Top-k 결과를 제공한다. 제안하는 기법의 우수성을 보이기 위하여 다양한 성능평가를 수행한다.

유사도 알고리즘을 활용한 시맨틱 프로세스 검색방안 (Semantic Process Retrieval with Similarity Algorithms)

  • 이홍주
    • Asia pacific journal of information systems
    • /
    • 제18권1호
    • /
    • pp.79-96
    • /
    • 2008
  • One of the roles of the Semantic Web services is to execute dynamic intra-organizational services including the integration and interoperation of business processes. Since different organizations design their processes differently, the retrieval of similar semantic business processes is necessary in order to support inter-organizational collaborations. Most approaches for finding services that have certain features and support certain business processes have relied on some type of logical reasoning and exact matching. This paper presents our approach of using imprecise matching for expanding results from an exact matching engine to query the OWL(Web Ontology Language) MIT Process Handbook. MIT Process Handbook is an electronic repository of best-practice business processes. The Handbook is intended to help people: (1) redesigning organizational processes, (2) inventing new processes, and (3) sharing ideas about organizational practices. In order to use the MIT Process Handbook for process retrieval experiments, we had to export it into an OWL-based format. We model the Process Handbook meta-model in OWL and export the processes in the Handbook as instances of the meta-model. Next, we need to find a sizable number of queries and their corresponding correct answers in the Process Handbook. Many previous studies devised artificial dataset composed of randomly generated numbers without real meaning and used subjective ratings for correct answers and similarity values between processes. To generate a semantic-preserving test data set, we create 20 variants for each target process that are syntactically different but semantically equivalent using mutation operators. These variants represent the correct answers of the target process. We devise diverse similarity algorithms based on values of process attributes and structures of business processes. We use simple similarity algorithms for text retrieval such as TF-IDF and Levenshtein edit distance to devise our approaches, and utilize tree edit distance measure because semantic processes are appeared to have a graph structure. Also, we design similarity algorithms considering similarity of process structure such as part process, goal, and exception. Since we can identify relationships between semantic process and its subcomponents, this information can be utilized for calculating similarities between processes. Dice's coefficient and Jaccard similarity measures are utilized to calculate portion of overlaps between processes in diverse ways. We perform retrieval experiments to compare the performance of the devised similarity algorithms. We measure the retrieval performance in terms of precision, recall and F measure? the harmonic mean of precision and recall. The tree edit distance shows the poorest performance in terms of all measures. TF-IDF and the method incorporating TF-IDF measure and Levenshtein edit distance show better performances than other devised methods. These two measures are focused on similarity between name and descriptions of process. In addition, we calculate rank correlation coefficient, Kendall's tau b, between the number of process mutations and ranking of similarity values among the mutation sets. In this experiment, similarity measures based on process structure, such as Dice's, Jaccard, and derivatives of these measures, show greater coefficient than measures based on values of process attributes. However, the Lev-TFIDF-JaccardAll measure considering process structure and attributes' values together shows reasonably better performances in these two experiments. For retrieving semantic process, we can think that it's better to consider diverse aspects of process similarity such as process structure and values of process attributes. We generate semantic process data and its dataset for retrieval experiment from MIT Process Handbook repository. We suggest imprecise query algorithms that expand retrieval results from exact matching engine such as SPARQL, and compare the retrieval performances of the similarity algorithms. For the limitations and future work, we need to perform experiments with other dataset from other domain. And, since there are many similarity values from diverse measures, we may find better ways to identify relevant processes by applying these values simultaneously.

Modern Methods of Text Analysis as an Effective Way to Combat Plagiarism

  • Myronenko, Serhii;Myronenko, Yelyzaveta
    • International Journal of Computer Science & Network Security
    • /
    • 제22권8호
    • /
    • pp.242-248
    • /
    • 2022
  • The article presents the analysis of modern methods of automatic comparison of original and unoriginal text to detect textual plagiarism. The study covers two types of plagiarism - literal, when plagiarists directly make exact copying of the text without changing anything, and intelligent, using more sophisticated techniques, which are harder to detect due to the text manipulation, like words and signs replacement. Standard techniques related to extrinsic detection are string-based, vector space and semantic-based. The first, most common and most successful target models for detecting literal plagiarism - N-gram and Vector Space are analyzed, and their advantages and disadvantages are evaluated. The most effective target models that allow detecting intelligent plagiarism, particularly identifying paraphrases by measuring the semantic similarity of short components of the text, are investigated. Models using neural network architecture and based on natural language sentence matching approaches such as Densely Interactive Inference Network (DIIN), Bilateral Multi-Perspective Matching (BiMPM) and Bidirectional Encoder Representations from Transformers (BERT) and its family of models are considered. The progress in improving plagiarism detection systems, techniques and related models is summarized. Relevant and urgent problems that remain unresolved in detecting intelligent plagiarism - effective recognition of unoriginal ideas and qualitatively paraphrased text - are outlined.

Ontology Matching Method Based on Word Embedding and Structural Similarity

  • Hongzhou Duan;Yuxiang Sun;Yongju Lee
    • International journal of advanced smart convergence
    • /
    • 제12권3호
    • /
    • pp.75-88
    • /
    • 2023
  • In a specific domain, experts have different understanding of domain knowledge or different purpose of constructing ontology. These will lead to multiple different ontologies in the domain. This phenomenon is called the ontology heterogeneity. For research fields that require cross-ontology operations such as knowledge fusion and knowledge reasoning, the ontology heterogeneity has caused certain difficulties for research. In this paper, we propose a novel ontology matching model that combines word embedding and a concatenated continuous bag-of-words model. Our goal is to improve word vectors and distinguish the semantic similarity and descriptive associations. Moreover, we make the most of textual and structural information from the ontology and external resources. We represent the ontology as a graph and use the SimRank algorithm to calculate the structural similarity. Our approach employs a similarity queue to achieve one-to-many matching results which provide a wider range of insights for subsequent mining and analysis. This enhances and refines the methodology used in ontology matching.

Word2Vec 기반의 의미적 유사도를 고려한 웹사이트 키워드 선택 기법 (Web Site Keyword Selection Method by Considering Semantic Similarity Based on Word2Vec)

  • 이동훈;김관호
    • 한국전자거래학회지
    • /
    • 제23권2호
    • /
    • pp.83-96
    • /
    • 2018
  • 문서를 대표하는 키워드를 추출하는 것은 문서의 정보를 빠르게 전달할 수 있을 뿐만 아니라 문서의 검색, 분류, 추천시스템 등의 자동화서비스에 유용하게 사용 될 수 있어 매우 중요하다. 그러나 웹사이트 문서에서 출현하는 단어의 빈도수, 단어의 동시출현관계를 통한 그래프 알고리즘 등의 기반으로 키워드를 추출할 경우 웹페이지 구조상 잠재적으로 주제와 관련이 없는 다양한 단어를 포함하고 있는 문제점과 한국어 형태소 분석의 정확성이 떨어지는 형태소 분석기 성능의 한계점 때문에 의미적인 키워드를 추출하는데 어려움이 존재한다. 따라서 본 논문에서는 의미적 단어 위주로 구축된 후보키워드들의 집합과 의미적 유사도 기반의 후보 키워드를 선택하는 방법으로써 의미적 키워드를 추출하지 못하는 문제점과 형태소 분석의 정확성이 떨어지는 문제점을 해결하고 일관성 없는 키워드를 제거하는 필터링 과정을 통해 최종 의미적 키워드를 추출하는 기법을 제안한다. 실 중소기업 웹페이지를 통한 실험 결과, 본 연구에서 제안한 기법의 성능이 통계적 유사도 기반의 키워드 선택기법보다 34.52% 향상된 것을 확인하였다. 따라서 단어 간의 의미적 유사성을 고려하고 일관성 없는 키워드를 제거함으로써 문서에서 키워드를 추출하는 성능을 향상시켰음을 확인하였다.

Question Similarity Measurement of Chinese Crop Diseases and Insect Pests Based on Mixed Information Extraction

  • Zhou, Han;Guo, Xuchao;Liu, Chengqi;Tang, Zhan;Lu, Shuhan;Li, Lin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제15권11호
    • /
    • pp.3991-4010
    • /
    • 2021
  • The Question Similarity Measurement of Chinese Crop Diseases and Insect Pests (QSM-CCD&IP) aims to judge the user's tendency to ask questions regarding input problems. The measurement is the basis of the Agricultural Knowledge Question and Answering (Q & A) system, information retrieval, and other tasks. However, the corpus and measurement methods available in this field have some deficiencies. In addition, error propagation may occur when the word boundary features and local context information are ignored when the general method embeds sentences. Hence, these factors make the task challenging. To solve the above problems and tackle the Question Similarity Measurement task in this work, a corpus on Chinese crop diseases and insect pests(CCDIP), which contains 13 categories, was established. Then, taking the CCDIP as the research object, this study proposes a Chinese agricultural text similarity matching model, namely, the AgrCQS. This model is based on mixed information extraction. Specifically, the hybrid embedding layer can enrich character information and improve the recognition ability of the model on the word boundary. The multi-scale local information can be extracted by multi-core convolutional neural network based on multi-weight (MM-CNN). The self-attention mechanism can enhance the fusion ability of the model on global information. In this research, the performance of the AgrCQS on the CCDIP is verified, and three benchmark datasets, namely, AFQMC, LCQMC, and BQ, are used. The accuracy rates are 93.92%, 74.42%, 86.35%, and 83.05%, respectively, which are higher than that of baseline systems without using any external knowledge. Additionally, the proposed method module can be extracted separately and applied to other models, thus providing reference for related research.

Word2Vec 학습을 통한 의미 기반 해외 유사 특허 검색 방안 (Identifying Similar Overseas Patent Using Word2Vec-Based Semantic Text Analytics)

  • 백민지;김남규
    • 한국IT서비스학회지
    • /
    • 제17권2호
    • /
    • pp.129-142
    • /
    • 2018
  • Recently, the number of patent applications have been increasing rapidly every year as the importance of protecting intellectual property rights becomes more important. Patents must be inventive and have novelty. Especially, the novelty implies that the corresponding invention is not the same as the previous invention. To confirm the novelty, prior art search must be conducted before and after the application. The target of prior art search should include not only Korean patents but also foreign patents. Search of foreign patents should be supported by multilingual search techniques. However, a dictionary-based naive approach shows a limitation because some technical concepts are represented in different terms according to each nation. For example, a Korean term and a Japanese term may not be synonym even though they represent the same technical concept. In this paper, we propose a new method to map semantic similarity between technical terms in Korean patents and Japanese patents. To investigate different representations in each nation for the same technical concept, we identified and analyzed pairs of patents those are mutually connected with priority claim relationship. By performing an experiment with real-world data, we showed that our approach can reveal semantically similar technical terms in other language successfully.

위키피디아를 이용한 분류자질 선정에 관한 연구 (An Experimental Study on Feature Selection Using Wikipedia for Text Categorization)

  • 김용환;정영미
    • 정보관리학회지
    • /
    • 제29권2호
    • /
    • pp.155-171
    • /
    • 2012
  • 텍스트 범주화에 있어서 일반적인 문제는 문헌을 표현하는 핵심적인 용어라도 학습문헌 집합에 나타나지 않으면 이 용어는 분류자질로 선정되지 않는다는 것과 형태가 다른 동의어들은 서로 다른 자질로 사용된다는 점이다. 이 연구에서는 위키피디아를 활용하여 문헌에 나타나는 동의어들을 하나의 분류자질로 변환하고, 학습문헌 집합에 출현하지 않은 입력문헌의 용어를 가장 유사한 학습문헌의 용어로 대체함으로써 범주화 성능을 향상시키고자 하였다. 분류자질 선정 실험에서는 (1) 비학습용어 추출 시 범주 정보의 사용여부, (2) 용어의 유사도 측정 방법(위키피디아 문서의 제목과 본문, 카테고리 정보, 링크 정보), (3) 유사도 척도(단순 공기빈도, 정규화된 공기빈도) 등 세 가지 조건을 결합하여 실험을 수행하였다. 비학습용어를 유사도 임계치 이상의 최고 유사도를 갖는 학습용어로 대체하여 kNN 분류기로 분류할 경우 모든 조건 결합에서 범주화 성능이 0.35%~1.85% 향상되었다. 실험 결과 범주화 성능이 크게 향상되지는 못하였지만 위키피디아를 활용하여 분류자질을 선정하는 방법이 효과적인 것으로 확인되었다.