• Title/Summary/Keyword: TextRank Algorithm

Search Result 27, Processing Time 0.021 seconds

Method of Extracting the Topic Sentence Considering Sentence Importance based on ELMo Embedding (ELMo 임베딩 기반 문장 중요도를 고려한 중심 문장 추출 방법)

  • Kim, Eun Hee;Lim, Myung Jin;Shin, Ju Hyun
    • Smart Media Journal
    • /
    • v.10 no.1
    • /
    • pp.39-46
    • /
    • 2021
  • This study is about a method of extracting a summary from a news article in consideration of the importance of each sentence constituting the article. We propose a method of calculating sentence importance by extracting the probabilities of topic sentence, similarity with article title and other sentences, and sentence position as characteristics that affect sentence importance. At this time, a hypothesis is established that the Topic Sentence will have a characteristic distinct from the general sentence, and a deep learning-based classification model is trained to obtain a topic sentence probability value for the input sentence. Also, using the pre-learned ELMo language model, the similarity between sentences is calculated based on the sentence vector value reflecting the context information and extracted as sentence characteristics. The topic sentence classification performance of the LSTM and BERT models was 93% accurate, 96.22% recall, and 89.5% precision, resulting in high analysis results. As a result of calculating the importance of each sentence by combining the extracted sentence characteristics, it was confirmed that the performance of extracting the topic sentence was improved by about 10% compared to the existing TextRank algorithm.

Analysis and Improvement of Ranking Algorithm for Web Mining System on the Hierarchical Web Environment

  • Heebyung Yoon;Lee, Kil-Seup;Kim, Hwa-Soo
    • Proceedings of the Korean Institute of Intelligent Systems Conference
    • /
    • 2003.09a
    • /
    • pp.455-458
    • /
    • 2003
  • The variety of document ranking algorithms have developed to provide efficient mining results for user's query on the web environment. The typical ranking algorithms are the Vector-Space Model based on the text, PsgeRank and HITS algorithms based on the hyperlink structures and other several improvement algorithms. All these are for the user's convenience and preference. However, these algorithms are usually developed on then Horizontal and non-hierarchial web environments and are not suitable for the hierarchial web environments such as enterprise and defense networks. Thus, we must consider the special environment factors in order to improve the ranking algorithms. In this paper, we analyze the several typical algorithms used by hyperlink structures on the web environment. We, then suggest a configuration of the hierarchical web environment and also give the relations between agents of the web mining system. Next, we propose an improved ranking algorithm suitable to this kind of special environments. The proposed algorithm is considered both the hyperlink structures of the documents and the location of the user of the hierarchical web.

  • PDF

User Reputation Evaluation Using Co-occurrence Feature and Collective Intelligence (동시출현 자질과 집단 지성을 이용한 지식검색 문서 사용자 명성 평가)

  • Lee, Hyun-Woo;Han, Yo-Sub;Kim, Lae-Hyun;Cha, Jeong-Won
    • Korean Journal of Cognitive Science
    • /
    • v.19 no.4
    • /
    • pp.459-476
    • /
    • 2008
  • The user needs to find the answer to your question is growing fast at the service using collective intelligent knowledge. In the previous researches, it was proven that the non-text information like view counting, referrer number, and number of answer is good in evaluating answers. There were also many works about evaluating answers using the various kinds of word dictionaries. In this work, we propose new method to evaluate answers to question effectively using user reputation that estimated by the social activity. We use a modified PageRank algorithm for estimating user reputation. We also use the similarity between question and answer. From the result of experiment in the Naver GisikiN corpus, we can see that the proposed method gives meaningful performance to complement the answer selection rate.

  • PDF

Ontology Matching Method Based on Word Embedding and Structural Similarity

  • Hongzhou Duan;Yuxiang Sun;Yongju Lee
    • International journal of advanced smart convergence
    • /
    • v.12 no.3
    • /
    • pp.75-88
    • /
    • 2023
  • In a specific domain, experts have different understanding of domain knowledge or different purpose of constructing ontology. These will lead to multiple different ontologies in the domain. This phenomenon is called the ontology heterogeneity. For research fields that require cross-ontology operations such as knowledge fusion and knowledge reasoning, the ontology heterogeneity has caused certain difficulties for research. In this paper, we propose a novel ontology matching model that combines word embedding and a concatenated continuous bag-of-words model. Our goal is to improve word vectors and distinguish the semantic similarity and descriptive associations. Moreover, we make the most of textual and structural information from the ontology and external resources. We represent the ontology as a graph and use the SimRank algorithm to calculate the structural similarity. Our approach employs a similarity queue to achieve one-to-many matching results which provide a wider range of insights for subsequent mining and analysis. This enhances and refines the methodology used in ontology matching.

Collection and Extraction Algorithm of Field-Associated Terms (분야연상어의 수집과 추출 알고리즘)

  • Lee, Sang-Kon;Lee, Wan-Kwon
    • The KIPS Transactions:PartB
    • /
    • v.10B no.3
    • /
    • pp.347-358
    • /
    • 2003
  • VSField-associated term is a single or compound word whose terms occur in any document, and which makes it possible to recognize a field of text by using common knowledge of human. For example, human recognizes the field of document such as or , a field name of text, when she encounters a word 'Pitcher' or 'election', respectively We Proposes an efficient construction method of field-associated terms (FTs) for specializing field to decide a field of text. We could fix document classification scheme from well-classified document database or corpus. Considering focus field we discuss levels and stability ranks of field-associated terms. To construct a balanced FT collection, we construct a single FTs. From the collections we could automatically construct FT's levels, and stability ranks. We propose a new extraction algorithms of FT's for document classification by using FT's concentration rate, its occurrence frequencies.

A Study of Programming Language Class with Lego NXT Robot for University of Education Students - Centered on Maze Problem - (레고 NXT 로봇을 활용한 예비교사의 프로그래밍 언어 수업 방안 - 미로 찾기 문제를 중심으로 -)

  • Hong, Ki-Cheon
    • Journal of The Korean Association of Information Education
    • /
    • v.13 no.1
    • /
    • pp.69-76
    • /
    • 2009
  • This paper proposes a teaching plan of programming language class for university of education students amusingly with LEGO Mindstorms NXT robot. The goal of class is not fragmentary knowledge acquirement but problem-solving of maze. This robot communicates with GUI named NXT-G installed in computer via USB. GUI is not text-based but icon-based programming tool. This paper designs a semester with 3 steps such as beginner, intermediate, high-rank. Beginner step is consists of learning of basic functions such as GUI usage and several sensors of robot. Intermediate step is consists of solving of maze problem with low complexity. High-rank step is consists of solving maze problem with medium and high complexity. All maze problem-solving have 3 process with algorithm, flowchart, and programming with stack.

  • PDF

A Research for Web Documents Genre Classification using STW (STW를 이용한 웹 문서 장르 분류에 관한 연구)

  • Ko, Byeong-Kyu;Oh, Kun-Seok;Kim, Pan-Koo
    • Journal of Information Technology and Architecture
    • /
    • v.9 no.4
    • /
    • pp.413-422
    • /
    • 2012
  • Many researchers have been studied to reveal human natural language to let machine understand its meaning by text based, page rank based or more. Particularly, it has been considered that URL and HTML Tag information in web documents are attracting people' attention again to analyze huge amount of web document automatically. In this paper, we propose a STW (Semantic Term Weight) approach based on syntactic and linguistic structure of web documents in order to classify what genres are. For the evaluation, we analyzed more than 1,000 documents from 20-Genre-collection corpus for training the documents based on SVM algorithm. Afterwards, we tested KI-04 corpus to evaluate performance of our proposed method. This paper measured their accuracy by classifying them into an experiment using STW and one without u sing STW. As the results, the proposed STW based approach showed approximately 10.2% which Is higher than one without use of STW.