• Title/Summary/Keyword: sentence summarization

Search Result 69, Processing Time 0.031 seconds

Text Summarization with Abductive Rules and Sentence Abstraction (개연성 규칙과 문장추상화를 활용한 문서요약)

  • Kim, Gon;Bae, Jae-Hak
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.11a
    • /
    • pp.359-362
    • /
    • 2002
  • 본 논문에서는 문장추상화와 문장간 개연적 연결상황을 활용한 문단 기준의 문서요약을 생각하였다. 구상한 문단기준 문서요약 방법론은 다음과 같은 절차로 구성되어 있다: (1) 문단의 문장들을 추상화시킨다, (2) 문장구성성분들의 문장간 개연적 연결상황을 확인한다, (3) 연결집중도가 상대적으로 높은 문장을 문단의 화제를 담고 있는 것으로 인정한다. 본 논문에서는 이 과정에서 문장추상화에 필요한 구문분석기와 온톨로지를 구체화하였고, 문장추상기로 설화문장 추상화를 하였다. 그 후 개연성 규칙을 적용하여 문단의 주제문을 선별하였다.

  • PDF

Korean Text Automatic Summarization using Semantically Expanded Sentence Similarity (의미적으로 확장된 문장 간 유사도를 이용한 한국어 텍스트 자동 요약)

  • Kim, Heechan;Lee, Soowon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2014.11a
    • /
    • pp.841-844
    • /
    • 2014
  • 텍스트 자동 요약은 수많은 텍스트 데이터를 처리함에 있어 중요한 연구 분야이다. 이중 추출요약은 현재 가장 많이 연구가 되고 있는 자동 요약 분야이다. 본 논문은 추출 요약의 선두 연구인 TextRank는 문장 간 유사도를 계산할 때 문장 내 단어 간의 의미적 유사성을 충분히 고려하지 못하였다. 본 연구에서는 의미적 유사성을 고려한 새로운 단어 간 유사도 측정 방법을 제안한다. 추출된 문장 간 유사도는 그래프로 표현되며, TextRank의 랭킹 알고리즘과 동일한 랭킹 알고리즘을 사용하여 실험적으로 평가하였다. 그 결과 문장 간 유사성을 고려할 때 단어의 의미적 요소를 충분히 고려하여 정보의 유실을 최소화하여야 한다는 것을 실험 결과로써 확인할 수 있었다.

Multi Sentence Summarization Method using Similarity Clustering of Word Embedding (워드 임베딩의 유사도 클러스터링을 통한 다중 문장 요약 생성 기법)

  • Lee, Pil-Won;Song, Jin-su;Shin, Yong-Tae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2021.05a
    • /
    • pp.290-292
    • /
    • 2021
  • 최근 인코더-디코더 구조의 자연어 처리모델이 활발하게 연구가 이루어지고 있다. 인코더-디코더기반의 언어모델은 특히 본문의 내용을 새로운 문장으로 요약하는 추상(Abstractive) 요약 분야에서 널리 사용된다. 그러나 기존의 언어모델은 단일 문서 및 문장을 전제로 설계되었기 때문에 기존의 언어모델에 다중 문장을 요약을 적용하기 어렵고 주제가 다양한 여러 문장을 요약하면 요약의 성능이 떨어지는 문제가 있다. 따라서 본 논문에서는 다중 문장으로 대표적이고 상품 리뷰를 워드 임베딩의 유사도를 기준으로 클러스터를 구성하여 관련성이 높은 문장 별로 인공 신경망 기반 언어모델을 통해 요약을 수행한다. 제안하는 모델의 성능을 평가하기 위해 전체 문장과 요약 문장의 유사도를 측정하여 요약문이 원문의 정보를 얼마나 포함하는지 실험한다. 실험 결과 기존의 RNN 기반의 요약 모델보다 뛰어난 성능의 요약을 수행했다.

BART with Random Sentence Insertion Noise for Korean Abstractive Summarization (무작위 문장 삽입 노이징을 적용한 BART 기반의 한국어 문서 추상 요약)

  • Park, Juhong;Kwon, Hongseok;Lee, Jong-Hyeok
    • Annual Conference on Human and Language Technology
    • /
    • 2020.10a
    • /
    • pp.455-458
    • /
    • 2020
  • 문서 요약은 입력 문서의 핵심 내용을 파악하여 짧고 간결한 문장으로 나타내는 과정이다. 최근에는 문서 요약을 위해 사전 학습된 언어 모델을 이용하는 방식이 여럿 제안되고 있지만, 이러한 언어 모델들은 문서 요약의 특성을 고려하지 않고 설계된 입력 노이즈 방식을 사용하는 한계점이 있다. 본 논문에서는 한국어 문서 추상 요약에 사전 학습 언어 모델인 BART를 도입하고, 입력 문서에 무작위 문장을 삽입하는 노이징 방식을 추가하여 문서 추상 요약 모델의 언어 이해 능력을 향상시키는 방법론을 제안한다. 실험 결과, BART를 도입한 문서 요약 모델의 결과는 다른 요약 모델들의 결과에 비해 전반적으로 품질 향상을 보였으며, BART와 함께 무작위 문장을 삽입하는 노이징 방법은 적은 비율로 삽입하는 경우 추가적인 성능 향상을 보였다.

  • PDF

Subject-Balanced Intelligent Text Summarization Scheme (주제 균형 지능형 텍스트 요약 기법)

  • Yun, Yeoil;Ko, Eunjung;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.2
    • /
    • pp.141-166
    • /
    • 2019
  • Recently, channels like social media and SNS create enormous amount of data. In all kinds of data, portions of unstructured data which represented as text data has increased geometrically. But there are some difficulties to check all text data, so it is important to access those data rapidly and grasp key points of text. Due to needs of efficient understanding, many studies about text summarization for handling and using tremendous amounts of text data have been proposed. Especially, a lot of summarization methods using machine learning and artificial intelligence algorithms have been proposed lately to generate summary objectively and effectively which called "automatic summarization". However almost text summarization methods proposed up to date construct summary focused on frequency of contents in original documents. Those summaries have a limitation for contain small-weight subjects that mentioned less in original text. If summaries include contents with only major subject, bias occurs and it causes loss of information so that it is hard to ascertain every subject documents have. To avoid those bias, it is possible to summarize in point of balance between topics document have so all subject in document can be ascertained, but still unbalance of distribution between those subjects remains. To retain balance of subjects in summary, it is necessary to consider proportion of every subject documents originally have and also allocate the portion of subjects equally so that even sentences of minor subjects can be included in summary sufficiently. In this study, we propose "subject-balanced" text summarization method that procure balance between all subjects and minimize omission of low-frequency subjects. For subject-balanced summary, we use two concept of summary evaluation metrics "completeness" and "succinctness". Completeness is the feature that summary should include contents of original documents fully and succinctness means summary has minimum duplication with contents in itself. Proposed method has 3-phases for summarization. First phase is constructing subject term dictionaries. Topic modeling is used for calculating topic-term weight which indicates degrees that each terms are related to each topic. From derived weight, it is possible to figure out highly related terms for every topic and subjects of documents can be found from various topic composed similar meaning terms. And then, few terms are selected which represent subject well. In this method, it is called "seed terms". However, those terms are too small to explain each subject enough, so sufficient similar terms with seed terms are needed for well-constructed subject dictionary. Word2Vec is used for word expansion, finds similar terms with seed terms. Word vectors are created after Word2Vec modeling, and from those vectors, similarity between all terms can be derived by using cosine-similarity. Higher cosine similarity between two terms calculated, higher relationship between two terms defined. So terms that have high similarity values with seed terms for each subjects are selected and filtering those expanded terms subject dictionary is finally constructed. Next phase is allocating subjects to every sentences which original documents have. To grasp contents of all sentences first, frequency analysis is conducted with specific terms that subject dictionaries compose. TF-IDF weight of each subjects are calculated after frequency analysis, and it is possible to figure out how much sentences are explaining about each subjects. However, TF-IDF weight has limitation that the weight can be increased infinitely, so by normalizing TF-IDF weights for every subject sentences have, all values are changed to 0 to 1 values. Then allocating subject for every sentences with maximum TF-IDF weight between all subjects, sentence group are constructed for each subjects finally. Last phase is summary generation parts. Sen2Vec is used to figure out similarity between subject-sentences, and similarity matrix can be formed. By repetitive sentences selecting, it is possible to generate summary that include contents of original documents fully and minimize duplication in summary itself. For evaluation of proposed method, 50,000 reviews of TripAdvisor are used for constructing subject dictionaries and 23,087 reviews are used for generating summary. Also comparison between proposed method summary and frequency-based summary is performed and as a result, it is verified that summary from proposed method can retain balance of all subject more which documents originally have.

Investigating an Automatic Method in Summarizing a Video Speech Using User-Assigned Tags (이용자 태그를 활용한 비디오 스피치 요약의 자동 생성 연구)

  • Kim, Hyun-Hee
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.46 no.1
    • /
    • pp.163-181
    • /
    • 2012
  • We investigated how useful video tags were in summarizing video speech and how valuable positional information was for speech summarization. Furthermore, we examined the similarity among sentences selected for a speech summary to reduce its redundancy. Based on such analysis results, we then designed and evaluated a method for automatically summarizing speech transcripts using a modified Maximum Marginal Relevance model. This model did not only reduce redundancy but it also enabled the use of social tags, title words, and sentence positional information. Finally, we compared the proposed method to the Extractor system in which key sentences of a video speech were chosen using the frequency and location information of speech content words. Results showed that the precision and recall rates of the proposed method were higher than those of the Extractor system, although there was no significant difference in the recall rates.

Product Evaluation Summarization Through Linguistic Analysis of Product Reviews (상품평의 언어적 분석을 통한 상품 평가 요약 시스템)

  • Lee, Woo-Chul;Lee, Hyun-Ah;Lee, Kong-Joo
    • The KIPS Transactions:PartB
    • /
    • v.17B no.1
    • /
    • pp.93-98
    • /
    • 2010
  • In this paper, we introduce a system that summarizes product evaluation through linguistic analysis to effectively utilize explosively increasing product reviews. Our system analyzes polarities of product reviews by product features, based on which customers evaluate each product like 'design' and 'material' for a skirt product category. The system shows to customers a graph as a review summary that represents percentages of positive and negative reviews. We build an opinion word dictionary for each product feature through context based automatic expansion with small seed words, and judge polarity of reviews by product features with the extracted dictionary. In experiment using product reviews from online shopping malls, our system shows average accuracy of 69.8% in extracting judgemental word dictionary and 81.8% in polarity resolution for each sentence.

Automatic Text Categorization using the Importance of Sentences (문장 중요도를 이용한 자동 문서 범주화)

  • Ko, Young-Joong;Park, Jin-Woo;Seo, Jung-Yun
    • Journal of KIISE:Software and Applications
    • /
    • v.29 no.6
    • /
    • pp.417-424
    • /
    • 2002
  • Automatic text categorization is a problem of assigning predefined categories to free text documents. In order to classify text documents, we have to extract good features from them. In previous researches, a text document is commonly represented by the frequency of each feature. But there is a difference between important and unimportant sentences in a text document. It has an effect on the importance of features in a text document. In this paper, we measure the importance of sentences in a text document using text summarizing techniques. A text document is represented by features with different weights according to the importance of each sentence. To verify the new method, we constructed Korean news group data set and experiment our method using it. We found that our new method gale a significant improvement over a basis system for our data sets.

Automatic Extraction of Paraphrases from a Parallel Bible Corpus (정렬된 성경 코퍼스로부터 바꿔쓰기표현(paraphrase)의 자동 추출)

  • Lee, Kong-Joo;Yun, Bo-Hyun
    • Korean Journal of Cognitive Science
    • /
    • v.17 no.4
    • /
    • pp.323-336
    • /
    • 2006
  • In this paper, we present a pilot system that can extract paraphrases from a parallel corpus using to-training method. Paraphrases are useful for the applications that should rreate a varied ind fluent text, such as machine translation, question-answering system, and multidocument summarization system. One of the difficulties in extracting paraphrases is to find a rich source from which we can extract paraphrases. The bible is one of the good sources fur extracting paraphrases as it has several Korean versions in which every sentence can be easily aligned by the chapter and the verse. We ran extract not only the lexical-level paraphrases but also the phrasal-level paraphrases from the parallel corpus which consists of the bibles using co-training method.

  • PDF

Investigating the Efficient Method for Constructing Audio Surrogates of Digital Video Data (비디오의 오디오 정보 요약 기법에 관한 연구)

  • Kim, Hyun-Hee
    • Journal of the Korean Society for information Management
    • /
    • v.26 no.3
    • /
    • pp.169-188
    • /
    • 2009
  • The study proposed the algorithm for automatically summarizing the audio information from a video and then conducted an experiment for the evaluation of the audio extraction that was constructed based on the proposed algorithm. The research results showed that first, the recall and precision rates of the proposed method for audio summarization were higher than those of the mechanical method by which audio extraction was constructed based on the sentence location. Second, the proposed method outperformed the mechanical method in summary making tasks, although in the gist recognition task(multiple choice), there is no statistically difference between the proposed and mechanical methods. In addition, the study conducted the participants' satisfaction survey regarding the use of audio extraction for video browsing and also discussed the practical implications of the proposed method in Internet and digital library environments.