• Title/Summary/Keyword: text extraction

Search Result 456, Processing Time 0.026 seconds

Automatic Construction of Reduced Dimensional Cluster-based Keyword Association Networks using LSI (LSI를 이용한 차원 축소 클러스터 기반 키워드 연관망 자동 구축 기법)

  • Yoo, Han-mook;Kim, Han-joon;Chang, Jae-young
    • Journal of KIISE
    • /
    • v.44 no.11
    • /
    • pp.1236-1243
    • /
    • 2017
  • In this paper, we propose a novel way of producing keyword networks, named LSI-based ClusterTextRank, which extracts significant key words from a set of clusters with a mutual information metric, and constructs an association network using latent semantic indexing (LSI). The proposed method reduces the dimension of documents through LSI, decomposes documents into multiple clusters through k-means clustering, and expresses the words within each cluster as a maximal spanning tree graph. The significant key words are identified by evaluating their mutual information within clusters. Then, the method calculates the similarities between the extracted key words using the term-concept matrix, and the results are represented as a keyword association network. To evaluate the performance of the proposed method, we used travel-related blog data and showed that the proposed method outperforms the existing TextRank algorithm by about 14% in terms of accuracy.

Logistic Regression Ensemble Method for Extracting Significant Information from Social Texts (소셜 텍스트의 주요 정보 추출을 위한 로지스틱 회귀 앙상블 기법)

  • Kim, So Hyeon;Kim, Han Joon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.5
    • /
    • pp.279-284
    • /
    • 2017
  • Currenty, in the era of big data, text mining and opinion mining have been used in many domains, and one of their most important research issues is to extract significant information from social media. Thus in this paper, we propose a logistic regression ensemble method of finding the main body text from blog HTML. First, we extract structural features and text features from blog HTML tags. Then we construct a classification model with logistic regression and ensemble that can decide whether any given tags involve main body text or not. One of our important findings is that the main body text can be found through 'depth' features extracted from HTML tags. In our experiment using diverse topics of blog data collected from the web, our tag classification model achieved 99% in terms of accuracy, and it recalled 80.5% of documents that have tags involving the main body text.

Comparison of term weighting schemes for document classification (문서 분류를 위한 용어 가중치 기법 비교)

  • Jeong, Ho Young;Shin, Sang Min;Choi, Yong-Seok
    • The Korean Journal of Applied Statistics
    • /
    • v.32 no.2
    • /
    • pp.265-276
    • /
    • 2019
  • The document-term frequency matrix is a general data of objects in text mining. In this study, we introduce a traditional term weighting scheme TF-IDF (term frequency-inverse document frequency) which is applied in the document-term frequency matrix and used for text classifications. In addition, we introduce and compare TF-IDF-ICSDF and TF-IGM schemes which are well known recently. This study also provides a method to extract keyword enhancing the quality of text classifications. Based on the keywords extracted, we applied support vector machine for the text classification. In this study, to compare the performance term weighting schemes, we used some performance metrics such as precision, recall, and F1-score. Therefore, we know that TF-IGM scheme provided high performance metrics and was optimal for text classification.

Deep Learning-based Text Summarization Model for Explainable Personalized Movie Recommendation Service (설명 가능한 개인화 영화 추천 서비스를 위한 딥러닝 기반 텍스트 요약 모델)

  • Chen, Biyao;Kang, KyungMo;Kim, JaeKyeong
    • Journal of Information Technology Services
    • /
    • v.21 no.2
    • /
    • pp.109-126
    • /
    • 2022
  • The number and variety of products and services offered by companies have increased dramatically, providing customers with more choices to meet their needs. As a solution to this information overload problem, the provision of tailored services to individuals has become increasingly important, and the personalized recommender systems have been widely studied and used in both academia and industry. Existing recommender systems face important problems in practical applications. The most important problem is that it cannot clearly explain why it recommends these products. In recent years, some researchers have found that the explanation of recommender systems may be very useful. As a result, users are generally increasing conversion rates, satisfaction, and trust in the recommender system if it is explained why those particular items are recommended. Therefore, this study presents a methodology of providing an explanatory function of a recommender system using a review text left by a user. The basic idea is not to use all of the user's reviews, but to provide them in a summarized form using only reviews left by similar users or neighbors involved in recommending the item as an explanation when providing the recommended item to the user. To achieve this research goal, this study aims to provide a product recommendation list using user-based collaborative filtering techniques, combine reviews left by neighboring users with each product to build a model that combines text summary methods among deep learning-based natural language processing methods. Using the IMDb movie database, text reviews of all target user neighbors' movies are collected and summarized to present descriptions of recommended movies. There are several text summary methods, but this study aims to evaluate whether the review summary is well performed by training the Sequence-to-sequence+attention model, which is a representative generation summary method, and the BertSum model, which is an extraction summary model.

HunMinJeomUm: Text Extraction and Braille Conversion System for the Learning of the Blind (시각장애인의 학습을 위한 텍스트 추출 및 점자 변환 시스템)

  • Kim, Chae-Ri;Kim, Ji-An;Kim, Yong-Min;Lee, Ye-Ji;Kong, Ki-Sok
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.21 no.5
    • /
    • pp.53-60
    • /
    • 2021
  • The number of visually impaired and blind people is increasing, but braille translation textbooks for them are insufficient, which violates their rights to education despite their will. In order to guarantee their rights, this paper develops a learning system, HunMinJeomUm, that helps them access textbooks, documents, and photographs that are not available in braille, without the assistance of others. In our system, a smart phone app and web pages are designed to promote the accessibility of the blind, and a braille kit is produced using Arduino and braille modules. The system supports the following functions. First, users select documents or pictures that they want, and the system extracts the text using OCR. Second, the extracted text is converted into voice and braille. Third, a membership registration function is provided so that the user can view the extracted text. Experiments have confirmed that our system generates braille and audio outputs successfully, and provides high OCR recognition rates. The study has also found that even completely blind users can easily access the smart phone app.

A Study on Rhythm Information Visualization Using Syllable of Digital Text (디지털 텍스트의 음절을 이용한 운율 정보 시각화에 관한 연구)

  • Park, seon-hee;Lee, jae-joong;Park, jin-wan
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2009.05a
    • /
    • pp.120-126
    • /
    • 2009
  • As the information age grows rapidly, the amount of digital texts has been increasing as well. It has brought an increasing of visualization case in order to figure out lots of digital texts. Existing visualized design of digital text is merely concentrating on figuration of subject word through adoption of stemming algorithm and word frequency extraction, prominence of meaning of text, and connection in between sentences. So it is a fact that expression of rhythm that can visualize sentimental feeing of digital text was insufficient. Syllable is a phoneme unit that can express rhythm more efficiently. In sentences, syllable is a most basic pronunciation unit in pronouncing word, phase and sentence. On this basis, accent, intonation, length of rhythm factor and others are based on syllable. Sonority, which is most closely associated with definitions of syllable, is expressed through air flow of igniting lung and acoustic energy that is specified kinetic energy into sonority. Seen from this perspective, this study examines phonologic definition and characteristics based on syllable, which is properties of digital text, and research the way to visualize rhythm through diagram. After converting digital text into phonetic symbol by the experiment, rhythm information are visualized into images using degree of resonance, which was started from rhythm in all languages, and using syllable establishment of digital text. By visualizing syllable information, it provides syllable information of digital text and express sentiment of digital text through diagram to assist user's understanding by systematic formula. Therefore, this study is aimed at planning for easy understanding of text's rhythm and realizing visualization of digital text.

  • PDF

Meta Learning based Global Relation Extraction trained by Traditional Korean data (전통 문화 데이터를 이용한 메타 러닝 기반 전역 관계 추출)

  • Kim, Kuekyeng;Kim, Gyeongmin;Jo, Jaechoon;Lim, Heuiseok
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.11
    • /
    • pp.23-28
    • /
    • 2018
  • Recent approaches to Relation Extraction methods mostly tend to be limited to mention level relation extractions. These types of methods, while featuring high performances, can only extract relations limited to a single sentence or so. The inability to extract these kinds of data is a terrible amount of information loss. To tackle this problem this paper presents an Augmented External Memory Neural Network model to enable Global Relation Extraction. the proposed model's Global relation extraction is done by first gathering and analyzing the mention level relation extraction by the Augmented External Memory. Additionally the proposed model shows high level of performances in korean due to the fact it can take the often omitted subjects and objectives into consideration.

Study on video character extraction and recognition (비디오 자막 추출 및 인식 기법에 관한 연구)

  • 김종렬;김성섭;문영식
    • Proceedings of the IEEK Conference
    • /
    • 2001.06c
    • /
    • pp.141-144
    • /
    • 2001
  • In this paper, a new algorithm for extracting and recognizing characters from video, without pre-knowledge such as font, color, size of character, is proposed. To improve the recognition rate for videos with complex background at low resolution, continuous frames with identical text region are automatically detected to compose an average frame. Using boundary pixels of a text region as seeds, we apply region filling to remove background from the character Then color clustering is applied to remove remaining backgrounds according to the verification of region filling process. Features such as white run and zero-one transition from the center, are extracted from unknown characters. These feature are compared with a pre-composed character feature set to recognize the characters.

  • PDF

The vectorization and recognition of circuit symbols for electronic circuit drawing management (전자회로 도면관리를 위한 벡터화와 회로 기호의 인식)

  • 백영묵;석종원;진성일;황찬식
    • Journal of the Korean Institute of Telematics and Electronics B
    • /
    • v.33B no.3
    • /
    • pp.176-185
    • /
    • 1996
  • Transformin the huge size of drawings into a suitable format for CAD system and recognizng the contents of drawings are the major concerans in the automated analysis of engineering drawings. This paper proposes some methods for text/graphics separation, symbol extraction, vectorization and symbol recognition with the object of applying them to electronic cirucit drawings. We use MBR (Minimum bounding rectangle) and size of isolated region on the drawings for separating text and graphic regions. Characteristics parameters such as the number of pixels, the length of circular constant and the degree of round shape are used for extracting loop symbols and geometric structures for non-loop symbols. To recognize symbols, nearest netighbor between FD (foruier descriptor) of extractd symbols and these of classification reference symbols is used. Experimental results show that the proposed method can generate compact vector representation of extracted symbols and perform the scale change and rotation of extracted symbol using symbol vectorization. Also we achieve an efficient searching of circuit drawings.

  • PDF

Segmentation of region strings using connection-characteristic function (연결특성함수를 이용한 문서화상에서의 영역 분리와 문자열 추출)

  • 김석태;이대원;박찬용;남궁재찬
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.22 no.11
    • /
    • pp.2531-2542
    • /
    • 1997
  • This paper describes a method for region segmentation and string extractionin documents which are mixed with text, graphic and picture images by the use of the structural characteristic of connceted components. In segmentation of non-text regionas, with connection-characteristic functions which are made by structural characteristic of connected components, segmentation process is progressed. In the string extraction, first we organize basic-unit-region of which vertical and horizontal length are 1/4 of average length of connection components. Second, by merging the basic-unit-regions one other that have smaller values than a given connection intensity threshold. Third, by linking the word blocks with similar block anagles, initial strings are cresed. Finally the whold strings are generated by merging remaining word blocks whose angles are not decided, if their height and prosition are similar to the initial strings. This method can extract strings that are neither horizontal nor of various character sizes. Through computer exteriments with different style documents, we have shown that the feasibility of our method successes.

  • PDF