• Title/Summary/Keyword: 질의응답문서

Search Result 163, Processing Time 0.023 seconds

QualityRank : Measuring Authority of Answer in Q&A Community using Social Network Analysis (QualityRank : 소셜 네트워크 분석을 통한 Q&A 커뮤니티에서 답변의 신뢰 수준 측정)

  • Kim, Deok-Ju;Park, Gun-Woo;Lee, Sang-Hoon
    • Journal of KIISE:Databases
    • /
    • v.37 no.6
    • /
    • pp.343-350
    • /
    • 2010
  • We can get answers we want to know via questioning in Knowledge Search Service (KSS) based on Q&A Community. However, it is getting more difficult to find credible documents in enormous documents, since many anonymous users regardless of credibility are participate in answering on the question. In previous works in KSS, researchers evaluated the quality of documents based on textual information, e.g. recommendation count, click count and non-textual information, e.g. answer length, attached data, conjunction count. Then, the evaluation results are used for enhancing search performance. However, the non-textual information has a problem that it is difficult to get enough information by users in the early stage of Q&A. The textual information also has a limitation for evaluating quality because of judgement by partial factors such as answer length, conjunction counts. In this paper, we propose the QualityRank algorithm to improve the problem by textual and non-textual information. This algorithm ranks the relevant and credible answers by considering textual/non-textual information and user centrality based on Social Network Analysis(SNA). Based on experimental validation we can confirm that the results by our algorithm is improved than those of textual/non-textual in terms of ranking performance.

Query-based Answer Extraction using Korean Dependency Parsing (의존 구문 분석을 이용한 질의 기반 정답 추출)

  • Lee, Dokyoung;Kim, Mintae;Kim, Wooju
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.3
    • /
    • pp.161-177
    • /
    • 2019
  • In this paper, we study the performance improvement of the answer extraction in Question-Answering system by using sentence dependency parsing result. The Question-Answering (QA) system consists of query analysis, which is a method of analyzing the user's query, and answer extraction, which is a method to extract appropriate answers in the document. And various studies have been conducted on two methods. In order to improve the performance of answer extraction, it is necessary to accurately reflect the grammatical information of sentences. In Korean, because word order structure is free and omission of sentence components is frequent, dependency parsing is a good way to analyze Korean syntax. Therefore, in this study, we improved the performance of the answer extraction by adding the features generated by dependency parsing analysis to the inputs of the answer extraction model (Bidirectional LSTM-CRF). The process of generating the dependency graph embedding consists of the steps of generating the dependency graph from the dependency parsing result and learning the embedding of the graph. In this study, we compared the performance of the answer extraction model when inputting basic word features generated without the dependency parsing and the performance of the model when inputting the addition of the Eojeol tag feature and dependency graph embedding feature. Since dependency parsing is performed on a basic unit of an Eojeol, which is a component of sentences separated by a space, the tag information of the Eojeol can be obtained as a result of the dependency parsing. The Eojeol tag feature means the tag information of the Eojeol. The process of generating the dependency graph embedding consists of the steps of generating the dependency graph from the dependency parsing result and learning the embedding of the graph. From the dependency parsing result, a graph is generated from the Eojeol to the node, the dependency between the Eojeol to the edge, and the Eojeol tag to the node label. In this process, an undirected graph is generated or a directed graph is generated according to whether or not the dependency relation direction is considered. To obtain the embedding of the graph, we used Graph2Vec, which is a method of finding the embedding of the graph by the subgraphs constituting a graph. We can specify the maximum path length between nodes in the process of finding subgraphs of a graph. If the maximum path length between nodes is 1, graph embedding is generated only by direct dependency between Eojeol, and graph embedding is generated including indirect dependencies as the maximum path length between nodes becomes larger. In the experiment, the maximum path length between nodes is adjusted differently from 1 to 3 depending on whether direction of dependency is considered or not, and the performance of answer extraction is measured. Experimental results show that both Eojeol tag feature and dependency graph embedding feature improve the performance of answer extraction. In particular, considering the direction of the dependency relation and extracting the dependency graph generated with the maximum path length of 1 in the subgraph extraction process in Graph2Vec as the input of the model, the highest answer extraction performance was shown. As a result of these experiments, we concluded that it is better to take into account the direction of dependence and to consider only the direct connection rather than the indirect dependence between the words. The significance of this study is as follows. First, we improved the performance of answer extraction by adding features using dependency parsing results, taking into account the characteristics of Korean, which is free of word order structure and omission of sentence components. Second, we generated feature of dependency parsing result by learning - based graph embedding method without defining the pattern of dependency between Eojeol. Future research directions are as follows. In this study, the features generated as a result of the dependency parsing are applied only to the answer extraction model in order to grasp the meaning. However, in the future, if the performance is confirmed by applying the features to various natural language processing models such as sentiment analysis or name entity recognition, the validity of the features can be verified more accurately.

A Semi-automatic Annotation Tool based on Named Entity Dictionary (개체명 사전 기반의 반자동 말뭉치 구축 도구)

  • Noh, Kyung-Mok;Kim, Chang-Hyun;Cheon, Min-Ah;Park, Ho-Min;Yoon, Ho;Kim, Jae-Kyun;Kim, Jae-Hoon
    • Annual Conference on Human and Language Technology
    • /
    • 2017.10a
    • /
    • pp.309-313
    • /
    • 2017
  • 개체명은 인명, 지명, 조직명 등 문서 내에서 중요한 의미를 가지므로 질의응답, 요약, 기계번역 분야에서 유용하게 사용되고 있다. 개체명 인식은 문서에서 개체명에 해당하는 단어를 찾아 개체명 범주를 부착하는 작업을 말한다. 개체명 인식 연구에는 개체명 범주가 부착된 개체명 말뭉치를 사용한다. 개체명의 범주는 연구 분야에 따라 다양하게 정의되므로 연구 분야에 적합한 개체명 말뭉치가 필요하다. 하지만 이런 말뭉치를 구축하는 일은 시간과 인력이 많이 필요하다. 따라서 본 논문에서는 개체명 사전 기반의 반자동 말뭉치 구축 도구를 제안한다. 제안하는 도구는 크게 전처리, 사용자 태깅, 후처리 단계로 나뉜다. 전처리 단계는 자동으로 개체명을 찾는 단계이다. 약 11만 개의 개체명을 기반으로 하여 트라이(trie) 구조의 개체명 사전을 구축한 후 사전을 이용하여 개체명을 자동으로 찾는다. 사용자 태깅 단계는 사용자가 수동으로 개체명을 태깅하는 단계이다. 전처리 단계에서 찾은 개체명 중 오류가 있는 개체명들은 수정하거나 삭제하고, 찾지 못한 개체명들은 사용자가 추가로 태깅하는 단계이다. 후처리 단계는 태깅한 결과로부터 사전 정보를 갱신하는 단계이다. 제안한 말뭉치 구축 도구를 이용하여 752개의 뉴스 기사에 대해 개체명을 태깅한 결과 7,620개의 개체명이 사전에 추가되었다. 제안한 도구를 사용한 결과 사용하지 않았을 때 비해 약 57.6% 정도 태깅 횟수가 감소했다.

  • PDF

Korean Coreference Resolution using the Multi-pass Sieve (Multi-pass Sieve를 이용한 한국어 상호참조해결)

  • Park, Cheon-Eum;Choi, Kyoung-Ho;Lee, Changki
    • Journal of KIISE
    • /
    • v.41 no.11
    • /
    • pp.992-1005
    • /
    • 2014
  • Coreference resolution finds all expressions that refer to the same entity in a document. Coreference resolution is important for information extraction, document classification, document summary, and question answering system. In this paper, we adapt Stanford's Multi-pass sieve system, the one of the best model of rule based coreference resolution to Korean. In this paper, all noun phrases are considered to mentions. Also, unlike Stanford's Multi-pass sieve system, the dependency parse tree is used for mention extraction, a Korean acronym list is built 'dynamically'. In addition, we propose a method that calculates weights by applying transitive properties of centers of the centering theory when refer Korean pronoun. The experiments show that our system obtains MUC 59.0%, $B_3$ 59.5%, Ceafe 63.5%, and CoNLL(Mean) 60.7%.

A Semi-automatic Annotation Tool based on Named Entity Dictionary (개체명 사전 기반의 반자동 말뭉치 구축 도구)

  • Noh, Kyung-Mok;Kim, Chang-Hyun;Cheon, Min-Ah;Park, Ho-Min;Yoon, Ho;Kim, Jae-Kyun;Kim, Jae-Hoon
    • 한국어정보학회:학술대회논문집
    • /
    • 2017.10a
    • /
    • pp.309-313
    • /
    • 2017
  • 개체명은 인명, 지명, 조직명 등 문서 내에서 중요한 의미를 가지므로 질의응답, 요약, 기계번역 분야에서 유용하게 사용되고 있다. 개체명 인식은 문서에서 개체명에 해당하는 단어를 찾아 개체명 범주를 부착하는 작업을 말한다. 개체명 인식 연구에는 개체명 범주가 부착된 개체명 말뭉치를 사용한다. 개체명의 범주는 연구 분야에 따라 다양하게 정의되므로 연구 분야에 적합한 개체명 말뭉치가 필요하다. 하지만 이런 말뭉치를 구축하는 일은 시간과 인력이 많이 필요하다. 따라서 본 논문에서는 개체명 사전 기반의 반자동 말뭉치 구축 도구를 제안한다. 제안하는 도구는 크게 전처리, 사용자 태깅, 후처리 단계로 나뉜다. 전처리 단계는 자동으로 개체명을 찾는 단계이다. 약 11만 개의 개체명을 기반으로 하여 트라이(trie) 구조의 개체명 사전을 구축한 후 사전을 이용하여 개체명을 자동으로 찾는다. 사용자 태깅 단계는 사용자가 수동으로 개체명을 태깅하는 단계이다. 전처리 단계에서 찾은 개체명 중 오류가 있는 개체명들은 수정하거나 삭제하고, 찾지 못한 개체명들은 사용자가 추가로 태깅하는 단계이다. 후처리 단계는 태깅한 결과로부터 사전 정보를 갱신하는 단계이다. 제안한 말뭉치 구축 도구를 이용하여 752개의 뉴스 기사에 대해 개체명을 태깅한 결과 7,620개의 개체명이 사전에 추가되었다. 제안한 도구를 사용한 결과 사용하지 않았을 때 비해 약 57.6% 정도 태깅 횟수가 감소했다.

  • PDF

A Study on the Integration of Recognition Technology for Scientific Core Entities (과학기술 핵심개체 인식기술 통합에 관한 연구)

  • Choi, Yun-Soo;Jeong, Chang-Hoo;Cho, Hyun-Yang
    • Journal of the Korean Society for information Management
    • /
    • v.28 no.1
    • /
    • pp.89-104
    • /
    • 2011
  • Large-scaled information extraction plays an important role in advanced information retrieval as well as question answering and summarization. Information extraction can be defined as a process of converting unstructured documents into formalized, tabular information, which consists of named-entity recognition, terminology extraction, coreference resolution and relation extraction. Since all the elementary technologies have been studied independently so far, it is not trivial to integrate all the necessary processes of information extraction due to the diversity of their input/output formation approaches and operating environments. As a result, it is difficult to handle scientific documents to extract both named-entities and technical terms at once. In order to extract these entities automatically from scientific documents at once, we developed a framework for scientific core entity extraction which embraces all the pivotal language processors, named-entity recognizer and terminology extractor.

Rule-based Normalization of Relative Temporal Information

  • Jeong, Young-Seob;Lim, Chaegyun;Lee, SeungDong;Mswahili, Medard Edmund;Ndomba, Goodwill Erasmo;Choi, Ho-Jin
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.12
    • /
    • pp.41-49
    • /
    • 2022
  • Documents often contain relative time expressions, and it is important to define a schema of the relative time information and develop a system that extracts such information from corpus. In this study, to deal with the relative time expressions, we propose seven additional attributes of timex3: year, month, day, week, hour, minute, and second. We propose a way to represent normalized values of the relative time expressions such as before, after, and count, and also design a set of rules to extract the relative time information from texts. With a new corpus constructed using the new attributes that consists of dialog, news, and history documents, we observed that our rule-set generally achieved 70% accuracy on the 1,041 documents. Especially, with the most frequently appeared attributes such as year, day, and week, we got higher accuracies compared to other attributes. The results of this study, our proposed timex3 attributes and the rule-set, will be useful in the development of services such as question-answer systems and chatbots.

A Survey on Deep Learning-based Pre-Trained Language Models (딥러닝 기반 사전학습 언어모델에 대한 이해와 현황)

  • Sangun Park
    • The Journal of Bigdata
    • /
    • v.7 no.2
    • /
    • pp.11-29
    • /
    • 2022
  • Pre-trained language models are the most important and widely used tools in natural language processing tasks. Since those have been pre-trained for a large amount of corpus, high performance can be expected even with fine-tuning learning using a small number of data. Since the elements necessary for implementation, such as a pre-trained tokenizer and a deep learning model including pre-trained weights, are distributed together, the cost and period of natural language processing has been greatly reduced. Transformer variants are the most representative pre-trained language models that provide these advantages. Those are being actively used in other fields such as computer vision and audio applications. In order to make it easier for researchers to understand the pre-trained language model and apply it to natural language processing tasks, this paper describes the definition of the language model and the pre-learning language model, and discusses the development process of the pre-trained language model and especially representative Transformer variants.

A Study on the Blockchain-based System Authentication Method (블록체인 기반 시스템 인증 방법에 대한 연구)

  • Kim, Sunghwan;Kim, Younggon
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.20 no.1
    • /
    • pp.211-218
    • /
    • 2020
  • Recently, with the advent of blockchain technology, attempts to apply this technology to existing systems are increasing. By using the blockchain technology consensus ledger and smart contract, it is necessary to distribute certificates to various fields that require documents, attestation, authentication, verification, etc. We are studying methods using hash operation, blockchain, etc., but it is difficult to spread the technology as it has not yet reached the stage of commercialization. In this paper, user device registration authentication algorithm, blockchain-based question and answer authentication algorithm, certificate issuance, verification process and encryption algorithm, and server-side authentication for easy application in blockchain based business platform environment We proposed a blockchain-based system authentication method using four algorithms.

Automatic Extraction of Paraphrases from a Parallel Bible Corpus (정렬된 성경 코퍼스로부터 바꿔쓰기표현(paraphrase)의 자동 추출)

  • Lee, Kong-Joo;Yun, Bo-Hyun
    • Korean Journal of Cognitive Science
    • /
    • v.17 no.4
    • /
    • pp.323-336
    • /
    • 2006
  • In this paper, we present a pilot system that can extract paraphrases from a parallel corpus using to-training method. Paraphrases are useful for the applications that should rreate a varied ind fluent text, such as machine translation, question-answering system, and multidocument summarization system. One of the difficulties in extracting paraphrases is to find a rich source from which we can extract paraphrases. The bible is one of the good sources fur extracting paraphrases as it has several Korean versions in which every sentence can be easily aligned by the chapter and the verse. We ran extract not only the lexical-level paraphrases but also the phrasal-level paraphrases from the parallel corpus which consists of the bibles using co-training method.

  • PDF