• 제목/요약/키워드: Length of Document

검색결과 76건 처리시간 0.024초

문서 길이 정규화를 이용한 문서 요약 자동화 시스템 구현 (Implementation of Text Summarize Automation Using Document Length Normalization)

  • 이재훈;김영천;이성주
    • 한국지능시스템학회:학술대회논문집
    • /
    • 한국퍼지및지능시스템학회 2001년도 추계학술대회 학술발표 논문집
    • /
    • pp.51-55
    • /
    • 2001
  • With the rapid growth of the World Wide Web and electronic information services, information is becoming available on-Line at an incredible rate. One result is the oft-decried information overload. No one has time to read everything, yet we often have to make critical decisions based on what we are able to assimilate. The technology of automatic text summarization is becoming indispensable for dealing with this problem. Text summarization is the process of distilling the most important information from a source to produce an abridged version for a particular user or task. Information retrieval(IR) is the task of searching a set of documents for some query-relevant documents. On the other hand, text summarization is considered to be the task of searching a document, a set of sentences, for some topic-relevant sentences. In this paper, we show that document information, that is more reliable and suitable for query, using document length normalization of which is gained through information retrieval . Experimental results of this system in newspaper articles show that document length normalization method superior to other methods use query itself.

  • PDF

문서영상의 에지 정보를 이용한 효과적인 블록분할 및 유형분류 (An Efficient Block Segmentation and Classification of a Document Image Using Edge Information)

  • 박창준;전준형;최형문
    • 전자공학회논문지B
    • /
    • 제33B권10호
    • /
    • pp.120-129
    • /
    • 1996
  • This paper presents an efficient block segmentation and classification using the edge information of the document image. We extract four prominent features form the edge gradient and orientaton, all of which, and thereby the block clssifications, are insensitive to the background noise and the brightness variation of of the image. Using these four features, we can efficiently classify a document image into the seven categrories of blocks of small-size letters, large-size letters, tables, equations, flow-charts, graphs, and photographs, the first five of which are text blocks which are character-recognizable, and the last two are non-character blocks. By introducing the clumn interval and text line intervals of the document in the determination of th erun length of CRLA (constrained run length algorithm), we can obtain an efficient block segmentation with reduced memory size. The simulation results show that the proposed algorithm can rigidly segment and classify the blocks of the documents into the above mentioned seven categories and classification performance is high enough for all the categories except for the graphs with too much variations.

  • PDF

Document Summarization via Convex-Concave Programming

  • Kim, Minyoung
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • 제16권4호
    • /
    • pp.293-298
    • /
    • 2016
  • Document summarization is an important task in various areas where the goal is to select a few the most descriptive sentences from a given document as a succinct summary. Even without training data of human labeled summaries, there has been several interesting existing work in the literature that yields reasonable performance. In this paper, within the same unsupervised learning setup, we propose a more principled learning framework for the document summarization task. Specifically we formulate an optimization problem that expresses the requirements of both faithful preservation of the document contents and the summary length constraint. We circumvent the difficult integer programming originating from binary sentence selection via continuous relaxation and the low entropy penalization. We also suggest an efficient convex-concave optimization solver algorithm that guarantees to improve the original objective at every iteration. For several document datasets, we demonstrate that the proposed learning algorithm significantly outperforms the existing approaches.

Automatic Single Document Text Summarization Using Key Concepts in Documents

  • Sarkar, Kamal
    • Journal of Information Processing Systems
    • /
    • 제9권4호
    • /
    • pp.602-620
    • /
    • 2013
  • Many previous research studies on extractive text summarization consider a subset of words in a document as keywords and use a sentence ranking function that ranks sentences based on their similarities with the list of extracted keywords. But the use of key concepts in automatic text summarization task has received less attention in literature on summarization. The proposed work uses key concepts identified from a document for creating a summary of the document. We view single-word or multi-word keyphrases of a document as the important concepts that a document elaborates on. Our work is based on the hypothesis that an extract is an elaboration of the important concepts to some permissible extent and it is controlled by the given summary length restriction. In other words, our method of text summarization chooses a subset of sentences from a document that maximizes the important concepts in the final summary. To allow diverse information in the summary, for each important concept, we select one sentence that is the best possible elaboration of the concept. Accordingly, the most important concept will contribute first to the summary, then to the second best concept, and so on. To prove the effectiveness of our proposed summarization method, we have compared it to some state-of-the art summarization systems and the results show that the proposed method outperforms the existing systems to which it is compared.

Category Factor Based Feature Selection for Document Classification

  • Kang Yun-Hee
    • International Journal of Contents
    • /
    • 제1권2호
    • /
    • pp.26-30
    • /
    • 2005
  • According to the fast growth of information on the Internet, it is becoming increasingly difficult to find and organize useful information. To reduce information overload, it needs to exploit automatic text classification for handling enormous documents. Support Vector Machine (SVM) is a model that is calculated as a weighted sum of kernel function outputs. This paper describes a document classifier for web documents in the fields of Information Technology and uses SVM to learn a model, which is constructed from the training sets and its representative terms. The basic idea is to exploit the representative terms meaning distribution in coherent thematic texts of each category by simple statistics methods. Vector-space model is applied to represent documents in the categories by using feature selection scheme based on TFiDF. We apply a category factor which represents effects in category of any term to the feature selection. Experiments show the results of categorization and the correlation of vector length.

  • PDF

동적 연결 그래프를 이용한 자동 문서 요약 시스템 (A Document Summarization System Using Dynamic Connection Graph)

  • 송원문;김영진;김은주;김명원
    • 한국정보과학회논문지:소프트웨어및응용
    • /
    • 제36권1호
    • /
    • pp.62-69
    • /
    • 2009
  • 문서 요약은 쉽고 빠르게 문서의 내용을 파악할 수 있도록 방대한 내용을 가지는 다양한 형태의 문서로부터 핵심 내용만을 추출하거나 생성하여 제공하는 것을 목적으로 한다. 본 논문에서는 효율적 문서 요약을 위해 주어진 문서의 평균 문장 길이(핵심어 개수)를 고려하여 문장 간의 핵심어 유사도를 나타내는 연결 그래프를 생성하고 분석하여 요약을 생성하는 기법을 제안한다. 또한 이러한 기법을 이용하여 응용 프로그램 문서로부터 자동으로 요약을 생성하는 자동 문서 요약 시스템을 개발한다. 제안한 방법의 객관적인 요약 성능 측정을 위해 정확한 요약문이 실린 20개의 테스트 문서를 이용하여 생성된 요약에 대해 precision(정확률)과 recall(재현율), F-measure를 측정하였으며, 실험 결과를 통해 기존 기법에 비해 우수한 요약 성능을 보임을 증명하였다.

Local Similarity based Document Layout Analysis using Improved ARLSA

  • Kim, Gwangbok;Kim, SooHyung;Na, InSeop
    • International Journal of Contents
    • /
    • 제11권2호
    • /
    • pp.15-19
    • /
    • 2015
  • In this paper, we propose an efficient document layout analysis algorithm that includes table detection. Typical methods of document layout analysis use the height and gap between words or columns. To correspond to the various styles and sizes of documents, we propose an algorithm that uses the mean value of the distance transform representing thickness and compare with components in the local area. With this algorithm, we combine a table detection algorithm using the same feature as that of the text classifier. Table candidates, separators, and big components are isolated from the image using Connected Component Analysis (CCA) and distance transform. The key idea of text classification is that the characteristics of the text parallel components that have a similar thickness and height. In order to estimate local similarity, we detect a text region using an adaptive searching window size. An improved adaptive run-length smoothing algorithm (ARLSA) was proposed to create the proper boundary of a text zone and non-text zone. Results from experiments on the ICDAR2009 page segmentation competition test set and our dataset demonstrate the superiority of our dataset through f-measure comparison with other algorithms.

Laser를 이용한 후물 판재류 및 Slap폭 측정 적용 (Application of slab and plate width measurement using laser distance meter)

  • 최철호
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 제어로봇시스템학회 1996년도 한국자동제어학술회의논문집(국내학술편); 포항공과대학교, 포항; 24-26 Oct. 1996
    • /
    • pp.1076-1078
    • /
    • 1996
  • According to ascending needs for quality assurance of plate by customer and automated measurement of dimension for slab, in plate mill, project is on preceeding to install measuring system to measure width, length and camber of slab and plate using laser distance meter. In this document, I will describe not technical point of view but idea of design for installation system.

  • PDF

Tax Avoidance and the Readability of Financial Statements: Empirical Evidence from Indonesia

  • PRATAMA, Bima Yoga;NARSA, Niluh Putu Dian Rosalina Handayani;PRANANJAYA, Kadek Pranetha
    • The Journal of Asian Finance, Economics and Business
    • /
    • 제9권2호
    • /
    • pp.103-112
    • /
    • 2022
  • This study aims to obtain empirical evidence regarding the link between tax avoidance (TA) and the readability of financial statements. This is a quantitative research using Ordinary Least Squares regression analysis which is then processed using STATA 14.0. A total of 278 companies listed on the Indonesia Stock Exchange during the period 2017-2019 is the data of this study. In detecting TA in a company, this study uses the ETR and CashETR and for the measurement of financial statement readability, this study uses gunning fog index and length of the document. The findings of this study suggest that tax avoidance and clear financial statements are mutually exclusive in the sense that when tax avoidance is practiced, companies will tend to conceal the information conveyed by financial statements. In other words, it is concluded that the more a company engages in tax avoidance, the lower the readability of the company's financial statements. This study provides in-depth evidence that tax avoidance is indirectly related to the disclosure of information by the company. Users of financial statements will realize that the company seeks to make disclosures that are in their best interests to avoid their tax avoidance strategy being detected.

문서 확장을 이용한 표제어 검색시스템 (Headword Finding System Using Document Expansion)

  • 김재훈;김형철
    • 정보관리연구
    • /
    • 제42권4호
    • /
    • pp.137-154
    • /
    • 2011
  • 표제어 검색시스템은 뜻풀이를 질의로 간주하는 정보검색 시스템이다. 이러한 시스템을 구축하기 위한 가장 간단한 방법으로 사전의 표제어 뜻풀이(사전 뜻풀이)를 문서로 간주하는 정보검색 시스템을 구축하는 것이다. 이 문서의 길이가 너무 짧아 사용자 질의(사용자 뜻풀이)에 대한 적절한 표제어를 검색하기 어렵다. 이 문제를 완화하기 위해서 본 논문에서는 정보검색에서 사용되는 질의 확장 개념을 문서 확장에 적용한다. 본 논문에서는 문서 확장 방법으로는 뜻풀이 확장과 유의어 확장을 사용한다. 뜻풀이 확장은 주어진 단어의 사전 뜻풀이에 속하는 단어의 뜻풀이를 문서에 포함시키는 방법이고, 유의어 확장은 무자질 군집화 알고리즘을 통해서 유의어를 찾고, 찾아진 유의어를 문서에 포함시키는 방법이다. 제안된 표제어 검색시스템은 사전 뜻풀이 그 자체를 입력으로 할 때, 16-포함률이 거의 100%에 달하였다. 또한 사용자 뜻풀이를 입력으로 할 때, 20-포함률이 66.9%였다. 사용자 뜻풀이가 단어의 의미를 충분히 전달할 수 없는 것으로 관찰되었으며 앞으로 정확하고 객관적인 평가를 위해서 평가 집합에 대한 연구가 추가적으로 필요한 실정이다.