• Title/Summary/Keyword: Keyword extraction

Search Result 188, Processing Time 0.036 seconds

스톰을 기반으로 한 실시간 SNS 데이터 분석 시스템

  • Lee, Hyeon-Gyeong;Go, Gi-Cheol;Son, Yeong-Seong;Kim, Jong-Bae
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2015.05a
    • /
    • pp.435-436
    • /
    • 2015
  • In order to analyze and maximize efficiency of advertise, business put more importance on SNS. Especially, keyword extraction analyses based on Hadoop receive attention. The existing keyword extraction analyses have mostly MapReduce processes. Due to that, it causes problems data base would not update in real time like SNS system. In this study, we indicate limitations of the existing model and suggest new model using Storm technique to analyze data in real time.

  • PDF

Document Content Similarity Detection Algorithm Using Word Cooccurrence Statistical Information Based Keyword Extraction (단어 공기 통계 정보 기반 색인어 추출을 활용한 문서 유사도 검사 알고리즘)

  • Kim, Jinkyu;Yi, Seungchul;Park, Kibong;Haing, Huhduck
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2016.01a
    • /
    • pp.111-113
    • /
    • 2016
  • 빠른 속도로 쏟아지고 있는 각종 발행물, 논문들에 대한 표절 검토는 표절 검출 알고리즘을 통해 직접적인 복제, 짜깁기, 말 바꾸어 쓰기 등을 검토하거나 표절 검토자가 직접 해당 문서의 키워드를 검색하여 확인하는 방식으로 이루어지고 있다. 하지만 점점 더 늘어나는 방대한 양의 문서들에 대한 표절 검토 작업은 더욱 정교한 검토 방법론을 필요로 하고 있으며, 이를 돕기 위해 문서의 직접적인 단어나 복제 비교에서 더 나아가 문서의 내용을 비교하여 비슷한 내용의 문서들을 필터링 및 검출할 수 있는 방법을 제안한다. 문서의 내용을 비교하기 위해 키워드 추출 알고리즘을 선행하며, 이를 통해 문서의 핵심 내용을 비교할 수 있는 기반을 마련하여 표절 검토자의 작업의 정확성과 속도를 향상시키고자 한다.

  • PDF

A Study of High Speed Retrieval Algorithm of Long Component Keyword (복합키워드의 고속검색 알고리즘에 관한 연구)

  • Lee Jin-Kwan;Jung Kyu-cheol;Lee Tae-hun;Park Ki-hong
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.8 no.8
    • /
    • pp.1769-1776
    • /
    • 2004
  • Effective keyword extraction is important in the information search system and there are several ways to select proper keyword in many keywords. Among them, DER Structure for AC Algorithm to search single keyword, can search multiple keywords but it has time complexity problem. In this paper, we developed a algorithm, "EDER structure" by expanding standalone search table based on DER structure search method to improve time complexity. We tested the algorithm using 500 text files and found that EDER structure is more efficient than DER structure for AC for keyword posting result and time complexity that 0.2 second for EDER and 0.6 second for DER structure,structure,

A Study of a Keyword Extraction System Design with a differentiated Service for Customer (고객 서비스 차별화를 위한 키워드 추출 시스템 설계에 관한 연구)

  • Lee, Hyun-Chang;Choi, Hyun-Seok;Shin, Sung-Yoon;Lee, Yang-Won
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2010.10a
    • /
    • pp.638-640
    • /
    • 2010
  • The business of companies has become more fierce due to the age of limitless competition. In addition, to improve the service satisfaction of their customers, they have continued their efforts. The researches about making their efforts get profits have done. As a result, CRM(customer relationship management) which is appropriate to analysis, evaluate and draw one of solutions to satisfy customers is received attention. Therefore, in this material we consider a architecture for keyword extraction system which is a kind of necessary technology for e-business model to provide a differentiated service. And then we could get the know-how of customer acquisition and maintenance.

  • PDF

A Study on the Optimal Search Keyword Extraction and Retrieval Technique Generation Using Word Embedding (워드 임베딩(Word Embedding)을 활용한 최적의 키워드 추출 및 검색 방법 연구)

  • Jeong-In Lee;Jin-Hee Ahn;Kyung-Taek Koh;YoungSeok Kim
    • Journal of the Korean Geosynthetics Society
    • /
    • v.22 no.2
    • /
    • pp.47-54
    • /
    • 2023
  • In this paper, we propose the technique of optimal search keyword extraction and retrieval for news article classification. The proposed technique was verified as an example of identifying trends related to North Korean construction. A representative Korean media platform, BigKinds, was used to select sample articles and extract keywords. The extracted keywords were vectorized using word embedding and based on this, the similarity between the extracted keywords was examined through cosine similarity. In addition, words with a similarity of 0.5 or higher were clustered based on the top 10 frequencies. Each cluster was formed as 'OR' between keywords inside the cluster and 'AND' between clusters according to the search form of the BigKinds. As a result of the in-depth analysis, it was confirmed that meaningful articles appropriate for the original purpose were extracted. This paper is significant in that it is possible to classify news articles suitable for the user's specific purpose without modifying the existing classification system and search form.

A Study on Graph-based Topic Extraction from Microblogs (마이크로블로그를 통한 그래프 기반의 토픽 추출에 관한 연구)

  • Choi, Don-Jung;Lee, Sung-Woo;Kim, Jae-Kwang;Lee, Jee-Hyong
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.21 no.5
    • /
    • pp.564-568
    • /
    • 2011
  • Microblogs became popular information delivery ways due to the spread of smart phones. They have the characteristic of reflecting the interests of users more quickly than other medium. Particularly, in case of the subject which attracts many users, microblogs can supply rich information originated from various information sources. Nevertheless, it has been considered as a hard problem to obtain useful information from microblogs because too much noises are in them. So far, various methods are proposed to extract and track some subjects from particular documents, yet these methods do not work effectively in case of microblogs which consist of short phrases. In this paper, we propose a graph-based topic extraction and partitioning method to understand interests of users about a certain keyword. The proposed method contains the process of generating a keyword graph using the co-occurrences of terms in the microblogs, and the process of splitting the graph by using a network partitioning method. When we applied the proposed method on some keywords. our method shows good performance for finding a topic about the keyword and partitioning the topic into sub-topics.

Hot Keyword Extraction of Sci-tech Periodicals Based on the Improved BERT Model

  • Liu, Bing;Lv, Zhijun;Zhu, Nan;Chang, Dongyu;Lu, Mengxin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.6
    • /
    • pp.1800-1817
    • /
    • 2022
  • With the development of the economy and the improvement of living standards, the hot issues in the subject area have become the main research direction, and the mining of the hot issues in the subject currently has problems such as a large amount of data and a complex algorithm structure. Therefore, in response to this problem, this study proposes a method for extracting hot keywords in scientific journals based on the improved BERT model.It can also provide reference for researchers,and the research method improves the overall similarity measure of the ensemble,introducing compound keyword word density, combining word segmentation, word sense set distance, and density clustering to construct an improved BERT framework, establish a composite keyword heat analysis model based on I-BERT framework.Taking the 14420 articles published in 21 kinds of social science management periodicals collected by CNKI(China National Knowledge Infrastructure) in 2017-2019 as the experimental data, the superiority of the proposed method is verified by the data of word spacing, class spacing, extraction accuracy and recall of hot keywords. In the experimental process of this research, it can be found that the method proposed in this paper has a higher accuracy than other methods in extracting hot keywords, which can ensure the timeliness and accuracy of scientific journals in capturing hot topics in the discipline, and finally pass Use information technology to master popular key words.

A study about IR Keyword Abstraction using AC Algorithm (AC 알고리즘을 이용한 정보검색 키워드 추출에 관한 연구)

  • 장혜숙;이진관;박기홍
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2002.11a
    • /
    • pp.667-671
    • /
    • 2002
  • It is very difficult to extract the words fitted for the purpose in spite of the great importance of efficient keyword extraction in information retrieval systems because there are many compound words. For example, AC machine is not able to search compound keywords from a single keyword. The DER structure solves this problem, but there remains a problem that it takes too much time to search keywords. Therefore a DERtable structure based on these methods is proposed in this dissertation to solve the above problems in which method tables are added to the existing DER structure and utilized to search keywords.

  • PDF

Deep Learning Document Analysis System Based on Keyword Frequency and Section Centrality Analysis

  • Lee, Jongwon;Wu, Guanchen;Jung, Hoekyung
    • Journal of information and communication convergence engineering
    • /
    • v.19 no.1
    • /
    • pp.48-53
    • /
    • 2021
  • Herein, we propose a document analysis system that analyzes papers or reports transformed into XML(Extensible Markup Language) format. It reads the document specified by the user, extracts keywords from the document, and compares the frequency of keywords to extract the top-three keywords. It maintains the order of the paragraphs containing the keywords and removes duplicated paragraphs. The frequency of the top-three keywords in the extracted paragraphs is re-verified, and the paragraphs are partitioned into 10 sections. Subsequently, the importance of the relevant areas is calculated and compared. By notifying the user of areas with the highest frequency and areas with higher importance than the average frequency, the user can read only the main content without reading all the contents. In addition, the number of paragraphs extracted through the deep learning model and the number of paragraphs in a section of high importance are predicted.

Keyword Spotting on Hangul Document Images Using Character Feature Models (문자 별 특징 모델을 이용한 한글 문서 영상에서 키워드 검색)

  • Park, Sang-Cheol;Kim, Soo-Hyung;Choi, Deok-Jai
    • The KIPS Transactions:PartB
    • /
    • v.12B no.5 s.101
    • /
    • pp.521-526
    • /
    • 2005
  • In this Paper, we propose a keyword spotting system as an alternative to searching system for poor quality Korean document images and compare the Proposed system with an OCR-based document retrieval system. The system is composed of character segmentation, feature extraction for the query keyword, and word-to-word matching. In the character segmentation step, we propose an effective method to remove the connectivity between adjacent characters and a character segmentation method by making the variance of character widths minimum. In the query creation step, feature vector for the query is constructed by a combination of a character model by typeface. In the matching step, word-to-word matching is applied base on a character-to-character matching. We demonstrated that the proposed keyword spotting system is more efficient than the OCR-based one to search a keyword on the Korean document images, especially when the quality of documents is quite poor and point size is small.