• Title/Summary/Keyword: 자동 추출 문서 요약

Search Result 62, Processing Time 0.022 seconds

Automatic Extraction and Usage of Terminology Dictionary Based on Definitional Sentences Patterns in Technical Documents (기술문서 정의문 패턴을 이용한 전문용어사전 자동추출 및 활용방안)

  • Han, Hui-Jeong;Kim, Tae-Young;Doo, Hyo-Chul;Oh, Hyo-Jung
    • Journal of the Korean Society for information Management
    • /
    • v.34 no.4
    • /
    • pp.81-99
    • /
    • 2017
  • Technical documents are important research outputs generated by knowledge and information society. In order to properly use the technical documents properly, it is necessary to utilize advanced information processing techniques, such as summarization and information extraction. In this paper, to extract core information, we automatically extracted the terminologies and their definition based on definitional sentences patterns and the structure of technical documents. Based on this, we proposed the system to build a specialized terminology dictionary. And further we suggested the personalized services so that users can utilize the terminology dictionary in various ways as an knowledge memory. The results of this study will allow users to find up-to-date information faster and easier. In addition, providing a personalized terminology dictionary to users can maximize the value, usability, and retrieval efficiency of the dictionary.

Latent Keyphrase Extraction Using LDA Model (LDA 모델을 이용한 잠재 키워드 추출)

  • Cho, Taemin;Lee, Jee-Hyong
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.25 no.2
    • /
    • pp.180-185
    • /
    • 2015
  • As the number of document resources is continuously increasing, automatically extracting keyphrases from a document becomes one of the main issues in recent days. However, most previous works have tried to extract keyphrases from words in documents, so they overlooked latent keyphrases which did not appear in documents. Although latent keyphrases do not appear in documents, they can undertake an important role in text summarization and information retrieval because they implicate meaningful concepts or contents of documents. Also, they cover more than one fourth of the entire keyphrases in the real-world datasets and they can be utilized in short articles such as SNS which rarely have explicit keyphrases. In this paper, we propose a new approach that selects candidate keyphrases from the keyphrases of neighbor documents which are similar to the given document and evaluates the importance of the candidates with the individual words in the candidates. Experiment result shows that latent keyphrases can be extracted at a reasonable level.

Automatically Constructing English-Korean Parallel Corpus from Web Documents (웹 문서로부터 한영 병렬말뭉치의 자동 구축)

  • Seo, Hyung-Won;Kim, Hyung-Chul;Cho, Hee-Young;Kim, Jae-Hoon;Yang, Sung-Il
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2006.11a
    • /
    • pp.161-164
    • /
    • 2006
  • 인터넷이 발전하면서 웹에는 같은 내용을 다양한 언어로 표현한 문서들이 많이 존재한다. 이와 같은 웹 문서의 성질을 이용하여, 이 논문은 웹으로부터 수집된 병렬문서(parallel document)를 이용하여 한영 병렬말뭉치 구축 시스템을 설계하고 구현한다. 이 논문에서 구축과정을 요약하면 다음과 같다. 첫째, 웹 문서수집기를 이용해서 웹으로부터 한영 웹문서(html 문서)를 각각 수집한다. 둘째, 수집된 각 언어의 웹 문서에서 불필요한 내용(태그와 광고 문구 등)을 제거하여 문장을 추출하고, 추출된 문장을 단락단위로 정렬한다. 셋째, 단락단위로 정렬된 문서를 문장정렬(sentence alignment) 방법을 이용해서 문장을 정렬한다. 끝으로 정렬된 병렬문장을 단어 단위로 분리하여 병렬말뭉치를 구축한다. 이와 같은 방법으로 이 논문에서는 약 42만 5천 문장의 한영 병렬말뭉치를 구축하였다.

  • PDF

Development of Advanced Intelligent Table Search System (향상된 지능형 테이블 검색 시스템의 개발)

  • Han, Kee-Jun;Kim, Seong-Chan;Liu, Ying
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2012.06b
    • /
    • pp.405-407
    • /
    • 2012
  • 학술 문서 내에서 테이블은 실험 결과, 정의, 요약하는 정보들을 함축하여 사용자에게 제공하는 역할을 한다. 즉 이러한 테이블을 학술 문서 내에서 탐색, 추출하여 검색에 이용하는 것은 학술 문서의 이해를 돕는 것과 더불어 학술 문서를 사용자가 직접 작성할 때에도 비슷한 형태의 테이블을 참조하여 형식에 맞는 테이블을 작성하는 데에 도움을 준다. 따라서 본 연구는 이러한 다양한 목적의 테이블 검색을 지원하기 위하여 문서로부터 자동으로 적합한 키워드를 추출하고 이를 통하여 문서와 유사한 테이블, 문서 내 테이블과 유사한 형식의 테이블을 검색하는 데 적합한 새로운 지능형 테이블 검색 시스템을 제안하며 이를 통해 기존에 존재하는 테이블 검색 시스템 알고리즘들과 성능 비교를 통해 향후 테이블 기반 검색 시스템 발전 가능성을 제시한다.

Question and Answering System through Search Result Summarization of Q&A Documents (Q&A 문서의 검색 결과 요약을 활용한 질의응답 시스템)

  • Yoo, Dong Hyun;Lee, Hyun Ah
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.4
    • /
    • pp.149-154
    • /
    • 2014
  • A user should pick up relevant answers by himself from various search results when using user participation question answering community like Knowledge-iN. If refined answers are automatically provided, usability of question answering community must be improved. This paper divides questions in Q&A documents into 4 types(word, list, graph and text), then proposes summarizing methods for each question type using document statistics. Summarized answers for word, list and text type are obtained by question clustering and calculating scores for words using frequency, proximity and confidence of answers. Answers for graph type is shown by extracting user opinion from answers.

Automatic Quality Evaluation with Completeness and Succinctness for Text Summarization (완전성과 간결성을 고려한 텍스트 요약 품질의 자동 평가 기법)

  • Ko, Eunjung;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.2
    • /
    • pp.125-148
    • /
    • 2018
  • Recently, as the demand for big data analysis increases, cases of analyzing unstructured data and using the results are also increasing. Among the various types of unstructured data, text is used as a means of communicating information in almost all fields. In addition, many analysts are interested in the amount of data is very large and relatively easy to collect compared to other unstructured and structured data. Among the various text analysis applications, document classification which classifies documents into predetermined categories, topic modeling which extracts major topics from a large number of documents, sentimental analysis or opinion mining that identifies emotions or opinions contained in texts, and Text Summarization which summarize the main contents from one document or several documents have been actively studied. Especially, the text summarization technique is actively applied in the business through the news summary service, the privacy policy summary service, ect. In addition, much research has been done in academia in accordance with the extraction approach which provides the main elements of the document selectively and the abstraction approach which extracts the elements of the document and composes new sentences by combining them. However, the technique of evaluating the quality of automatically summarized documents has not made much progress compared to the technique of automatic text summarization. Most of existing studies dealing with the quality evaluation of summarization were carried out manual summarization of document, using them as reference documents, and measuring the similarity between the automatic summary and reference document. Specifically, automatic summarization is performed through various techniques from full text, and comparison with reference document, which is an ideal summary document, is performed for measuring the quality of automatic summarization. Reference documents are provided in two major ways, the most common way is manual summarization, in which a person creates an ideal summary by hand. Since this method requires human intervention in the process of preparing the summary, it takes a lot of time and cost to write the summary, and there is a limitation that the evaluation result may be different depending on the subject of the summarizer. Therefore, in order to overcome these limitations, attempts have been made to measure the quality of summary documents without human intervention. On the other hand, as a representative attempt to overcome these limitations, a method has been recently devised to reduce the size of the full text and to measure the similarity of the reduced full text and the automatic summary. In this method, the more frequent term in the full text appears in the summary, the better the quality of the summary. However, since summarization essentially means minimizing a lot of content while minimizing content omissions, it is unreasonable to say that a "good summary" based on only frequency always means a "good summary" in its essential meaning. In order to overcome the limitations of this previous study of summarization evaluation, this study proposes an automatic quality evaluation for text summarization method based on the essential meaning of summarization. Specifically, the concept of succinctness is defined as an element indicating how few duplicated contents among the sentences of the summary, and completeness is defined as an element that indicating how few of the contents are not included in the summary. In this paper, we propose a method for automatic quality evaluation of text summarization based on the concepts of succinctness and completeness. In order to evaluate the practical applicability of the proposed methodology, 29,671 sentences were extracted from TripAdvisor 's hotel reviews, summarized the reviews by each hotel and presented the results of the experiments conducted on evaluation of the quality of summaries in accordance to the proposed methodology. It also provides a way to integrate the completeness and succinctness in the trade-off relationship into the F-Score, and propose a method to perform the optimal summarization by changing the threshold of the sentence similarity.

Setences Extraction System using Automatic Division of Paragraph (단락 자동 구분을 통한 중요 문자 추출)

  • 김계성;이현주;정영규;서연경;손기준;이상조
    • Proceedings of the Korean Society for Cognitive Science Conference
    • /
    • 2000.06a
    • /
    • pp.233-237
    • /
    • 2000
  • 본 논문은 단락의 자동 구분을 통한 중요 문장 추출 시스템을 제안한다. 먼저 어휘의 재출현 여부와 어휘의 일치도, 어휘의 역할 변화를 파악하여 재출현 어휘에 대한 양상을 분석하고 이를 통하여 문장 간의 긴밀도를 정량적으로 계산한다. 다음으로 측정된 문장 간 긴밀도를 이용하여 사용자의 추출 범위에 따라 단락을 구분하고, 각 단락의 대표 문장을 선정하여 최종 요약문을 생성한다. 제안한 방법은 문서 제목, 문장의 위치, 수사 구조 등의 정보를 이용하지 않으며, 단순히 어휘의 출현 빈도만을 이용하던 기존의 통계적인 방법보다 질높은 요약문을 생성할 수 있다. 또한 제안한 방법론은 본 논문이 대상으로 삼고 있는 신문기사의 영역뿐만 아니라 다른 영역으로의 적용이 가능하다.

  • PDF

An automatic extraction of newspaper articles using expanded 5WlH (확장된 6하원칙을 이용한 신문기사 자동요약)

  • 윤재민;강인수;권오옥;배재학;이종혁
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2002.04b
    • /
    • pp.466-468
    • /
    • 2002
  • 본 논문은 신문기사를 추출(Extract)하는데 있어서, 신문기사의 구조적인 특징을 분석한 후, 6하원칙(5W1H)과 전문(Lead)과의 관계를 유추하여 신문기사를 효과적으로 요약하기 위한 방법과 알고리즘을 제안하였다. 본 연구에서는 먼저 신문기사에서 가장 중요한 단락인 전문에서 6하원칙의 각 구성성분을 추출하고, 본문에서는 전문에 나타난 6하원칙의 각 성분이 어떻게 재사용되며 강조되고 있는가를 파악하기 위래 제안된 방법과 알고리즘을 이용하여 중요한 문장을 추출하였다. 실험문서는 조선일보 웹사이트에서 제공하는 신문기사 100건을 대상으로 하였으며 요약율이 20%와 30%일 경우 제안한 방법의 정확률은 각각 82.4%와 74.1%로 기존의 전문기반(Lead-based)방법보다 3.6%와 6.3% 향상되었다.

  • PDF

Automatic Extraction of Paraphrases from a Parallel Bible Corpus (정렬된 성경 코퍼스로부터 바꿔쓰기표현(paraphrase)의 자동 추출)

  • Lee, Kong-Joo;Yun, Bo-Hyun
    • Korean Journal of Cognitive Science
    • /
    • v.17 no.4
    • /
    • pp.323-336
    • /
    • 2006
  • In this paper, we present a pilot system that can extract paraphrases from a parallel corpus using to-training method. Paraphrases are useful for the applications that should rreate a varied ind fluent text, such as machine translation, question-answering system, and multidocument summarization system. One of the difficulties in extracting paraphrases is to find a rich source from which we can extract paraphrases. The bible is one of the good sources fur extracting paraphrases as it has several Korean versions in which every sentence can be easily aligned by the chapter and the verse. We ran extract not only the lexical-level paraphrases but also the phrasal-level paraphrases from the parallel corpus which consists of the bibles using co-training method.

  • PDF

Automatic Summary Method of Linguistic Educational Video Using Multiple Visual Features (다중 비주얼 특징을 이용한 어학 교육 비디오의 자동 요약 방법)

  • Han Hee-Jun;Kim Cheon-Seog;Choo Jin-Ho;Ro Yong-Man
    • Journal of Korea Multimedia Society
    • /
    • v.7 no.10
    • /
    • pp.1452-1463
    • /
    • 2004
  • The requirement of automatic video summary is increasing as bi-directional broadcasting contents and various user requests and preferences for the bi -directional broadcast environment are increasing. Automatic video summary is needed for an efficient management and usage of many contents in service provider as well. In this paper, we propose a method to generate a content-based summary of linguistic educational videos automatically. First, shot-boundaries and keyframes are generated from linguistic educational video and then multiple(low-level) visual features are extracted. Next, the semantic parts (Explanation part, Dialog part, Text-based part) of the linguistic educational video are generated using extracted visual features. Lastly the XMI- document describing summary information is made based on HieraTchical Summary architecture oi MPEG-7 MDS (Multimedia I)escription Scheme). Experimental results show that our proposed algorithm provides reasonable performance for automatic summary of linguistic educational videos. We verified that the proposed method is useful ior video summary system to provide various services as well as management of educational contents.

  • PDF