• Title/Summary/Keyword: Text analysis

Search Result 3,326, Processing Time 0.034 seconds

An Analysis of Online Black Market: Using Data Mining and Social Network Analysis (온라인 해킹 불법 시장 분석: 데이터 마이닝과 소셜 네트워크 분석 활용)

  • Kim, Minsu;Kim, Hee-Woong
    • The Journal of Information Systems
    • /
    • v.29 no.2
    • /
    • pp.221-242
    • /
    • 2020
  • Purpose This study collects data of the recently activated online black market and analyzes it to present a specific method for preparing for a hacking attack. This study aims to make safe from the cyber attacks, including hacking, from the perspective of individuals and businesses by closely analyzing hacking methods and tools in a situation where they are easily shared. Design/methodology/approach To prepare for the hacking attack through the online black market, this study uses the routine activity theory to identify the opportunity factors of the hacking attack. Based on this, text mining and social network techniques are applied to reveal the most dangerous areas of security. It finds out suitable targets in routine activity theory through text mining techniques and motivated offenders through social network analysis. Lastly, the absence of guardians and the parts required by guardians are extracted using both analysis techniques simultaneously. Findings As a result of text mining, there was a large supply of hacking gift cards, and the demand to attack sites such as Amazon and Netflix was very high. In addition, interest in accounts and combos was in high demand and supply. As a result of social network analysis, users who actively share hacking information and tools can be identified. When these two analyzes were synthesized, it was found that specialized managers are required in the areas of proxy, maker and many managers are required for the buyer network, and skilled managers are required for the seller network.

Text Mining of Online News, Social Media, and Consumer Review on Artificial Intelligence Service (인공지능 서비스에 대한 온라인뉴스, 소셜미디어, 소비자리뷰 텍스트마이닝)

  • Li, Xu;Lim, Hyewon;Yeo, Harim;Hwang, Hyesun
    • Human Ecology Research
    • /
    • v.59 no.1
    • /
    • pp.23-43
    • /
    • 2021
  • This study looked through the text mining analysis to check the status of the virtual assistant service, and explore the needs of consumers, and present consumer-oriented directions. Trendup 4.0 was used to analyze the keywords of AI services in Online News and social media from 2016 to 2020. The R program was used to collect consumer comment data and implement Topic Modeling analysis. According to the analysis, the number of mentions of AI services in mass media and social media has steadily increased. The Sentimental Analysis showed consumers were feeling positive about AI services in terms of useful and convenient functional and emotional aspects such as pleasure and interest. However, consumers were also experiencing complexity and difficulty with AI services and had concerns and fears about the use of AI services in the early stages of their introduction. The results of the consumer review analysis showed that there were topics(Technical Requirements) related to technology and the access process for the AI services to be provided, and topics (Consumer Request) expressed negative feelings about AI services, and topics(Consumer Life Support Area) about specific functions in the use of AI services. Text mining analysis enable this study to confirm consumer expectations or concerns about AI service, and to examine areas of service support that consumers experienced. The review data on each platform also revealed that the potential needs of consumers could be met by expanding the scope of support services and applying platform-specific strengths to provide differentiated services.

A Study on the Use of Stopword Corpus for Cleansing Unstructured Text Data (비정형 텍스트 데이터 정제를 위한 불용어 코퍼스의 활용에 관한 연구)

  • Lee, Won-Jo
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.6
    • /
    • pp.891-897
    • /
    • 2022
  • In big data analysis, raw text data mostly exists in various unstructured data forms, so it becomes a structured data form that can be analyzed only after undergoing heuristic pre-processing and computer post-processing cleansing. Therefore, in this study, unnecessary elements are purified through pre-processing of the collected raw data in order to apply the wordcloud of R program, which is one of the text data analysis techniques, and stopwords are removed in the post-processing process. Then, a case study of wordcloud analysis was conducted, which calculates the frequency of occurrence of words and expresses words with high frequency as key issues. In this study, to improve the problems of the "nested stopword source code" method, which is the existing stopword processing method, using the word cloud technique of R, we propose the use of "general stopword corpus" and "user-defined stopword corpus" and conduct case analysis. The advantages and disadvantages of the proposed "unstructured data cleansing process model" are comparatively verified and presented, and the practical application of word cloud visualization analysis using the "proposed external corpus cleansing technique" is presented.

BERT & Hierarchical Graph Convolution Neural Network based Emotion Analysis Model (BERT 및 계층 그래프 컨볼루션 신경망 기반 감성분석 모델)

  • Zhang, Junjun;Shin, Jongho;An, Suvin;Park, Taeyoung;Noh, Giseop
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.10a
    • /
    • pp.34-36
    • /
    • 2022
  • In the existing text sentiment analysis models, the entire text is usually directly modeled as a whole, and the hierarchical relationship between text contents is less considered. However, in the practice of sentiment analysis, many texts are mixed with multiple emotions. If the semantic modeling of the whole is directly performed, it may increase the difficulty of the sentiment analysis model to judge the sentiment, making the model difficult to apply to the classification of mixed-sentiment sentences. Therefore, this paper proposes a sentiment analysis model BHGCN that considers the text hierarchy. In this model, the output of hidden states of each layer of BERT is used as a node, and a directed connection is made between the upper and lower layers to construct a graph network with a semantic hierarchy. The model not only pays attention to layer-by-layer semantics, but also pays attention to hierarchical relationships. Suitable for handling mixed sentiment classification tasks. The comparative experimental results show that the BHGCN model exhibits obvious competitive advantages.

  • PDF

Text Region Extraction using Pattern Histogram of Character-Edge Map in Natural Images (문자-에지 맵의 패턴 히스토그램을 이용한 자연이미지에서의 텍스트 영역 추출)

  • Park, Jong-Cheon;Hwang, Dong-Guk;Lee, Woo-Ram;Kwon, Kyo-Hyun;Jun, Byoung-Min
    • Proceedings of the KAIS Fall Conference
    • /
    • 2006.11a
    • /
    • pp.220-224
    • /
    • 2006
  • The text to be included in the natural images has many important information in the natural image. Therefore, if we can extract the text in natural images, It can be applied to many important applications. In this paper, we propose a text region extraction method using pattern histogram of character-edge map. We extract the edges with the Canny edge detector and creates 16 kind of edge map from an extracted edges. And then we make a character-edge map of 8 kinds that have a character feature with a combination of an edge map. We extract text region using 8 kinds of character-edge map and 16 kind of edge map. Verification of text candidate region uses analysis of a character-edge map pattern histogram and structural feature of text region. The method to propose experimented with various kind of the natural images. The proposed approach extracted text region from a natural images to have been composed of a complex background, various letters, various text colors effectively.

  • PDF

Text Augmentation Using Hierarchy-based Word Replacement

  • Kim, Museong;Kim, Namgyu
    • Journal of the Korea Society of Computer and Information
    • /
    • v.26 no.1
    • /
    • pp.57-67
    • /
    • 2021
  • Recently, multi-modal deep learning techniques that combine heterogeneous data for deep learning analysis have been utilized a lot. In particular, studies on the synthesis of Text to Image that automatically generate images from text are being actively conducted. Deep learning for image synthesis requires a vast amount of data consisting of pairs of images and text describing the image. Therefore, various data augmentation techniques have been devised to generate a large amount of data from small data. A number of text augmentation techniques based on synonym replacement have been proposed so far. However, these techniques have a common limitation in that there is a possibility of generating a incorrect text from the content of an image when replacing the synonym for a noun word. In this study, we propose a text augmentation method to replace words using word hierarchy information for noun words. Additionally, we performed experiments using MSCOCO data in order to evaluate the performance of the proposed methodology.

Development of Sentiment Analysis Model for the hot topic detection of online stock forums (온라인 주식 포럼의 핫토픽 탐지를 위한 감성분석 모형의 개발)

  • Hong, Taeho;Lee, Taewon;Li, Jingjing
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.1
    • /
    • pp.187-204
    • /
    • 2016
  • Document classification based on emotional polarity has become a welcomed emerging task owing to the great explosion of data on the Web. In the big data age, there are too many information sources to refer to when making decisions. For example, when considering travel to a city, a person may search reviews from a search engine such as Google or social networking services (SNSs) such as blogs, Twitter, and Facebook. The emotional polarity of positive and negative reviews helps a user decide on whether or not to make a trip. Sentiment analysis of customer reviews has become an important research topic as datamining technology is widely accepted for text mining of the Web. Sentiment analysis has been used to classify documents through machine learning techniques, such as the decision tree, neural networks, and support vector machines (SVMs). is used to determine the attitude, position, and sensibility of people who write articles about various topics that are published on the Web. Regardless of the polarity of customer reviews, emotional reviews are very helpful materials for analyzing the opinions of customers through their reviews. Sentiment analysis helps with understanding what customers really want instantly through the help of automated text mining techniques. Sensitivity analysis utilizes text mining techniques on text on the Web to extract subjective information in the text for text analysis. Sensitivity analysis is utilized to determine the attitudes or positions of the person who wrote the article and presented their opinion about a particular topic. In this study, we developed a model that selects a hot topic from user posts at China's online stock forum by using the k-means algorithm and self-organizing map (SOM). In addition, we developed a detecting model to predict a hot topic by using machine learning techniques such as logit, the decision tree, and SVM. We employed sensitivity analysis to develop our model for the selection and detection of hot topics from China's online stock forum. The sensitivity analysis calculates a sentimental value from a document based on contrast and classification according to the polarity sentimental dictionary (positive or negative). The online stock forum was an attractive site because of its information about stock investment. Users post numerous texts about stock movement by analyzing the market according to government policy announcements, market reports, reports from research institutes on the economy, and even rumors. We divided the online forum's topics into 21 categories to utilize sentiment analysis. One hundred forty-four topics were selected among 21 categories at online forums about stock. The posts were crawled to build a positive and negative text database. We ultimately obtained 21,141 posts on 88 topics by preprocessing the text from March 2013 to February 2015. The interest index was defined to select the hot topics, and the k-means algorithm and SOM presented equivalent results with this data. We developed a decision tree model to detect hot topics with three algorithms: CHAID, CART, and C4.5. The results of CHAID were subpar compared to the others. We also employed SVM to detect the hot topics from negative data. The SVM models were trained with the radial basis function (RBF) kernel function by a grid search to detect the hot topics. The detection of hot topics by using sentiment analysis provides the latest trends and hot topics in the stock forum for investors so that they no longer need to search the vast amounts of information on the Web. Our proposed model is also helpful to rapidly determine customers' signals or attitudes towards government policy and firms' products and services.

Analysis of English abstracts in Journal of the Korean Data & Information Science Society using topic models and social network analysis (토픽 모형 및 사회연결망 분석을 이용한 한국데이터정보과학회지 영문초록 분석)

  • Kim, Gyuha;Park, Cheolyong
    • Journal of the Korean Data and Information Science Society
    • /
    • v.26 no.1
    • /
    • pp.151-159
    • /
    • 2015
  • This article analyzes English abstracts of the articles published in Journal of the Korean Data & Information Science Society using text mining techniques. At first, term-document matrices are formed by various methods and then visualized by social network analysis. LDA (latent Dirichlet allocation) and CTM (correlated topic model) are also employed in order to extract topics from the abstracts. Performances of the topic models are compared via entropy for several numbers of topics and weighting methods to form term-document matrices.

Text Mining and Network Analysis of News Articles for Deriving Socio-Economic Damage Types of Heat Wave Events in Korea: 2012~2016 Cases (뉴스 기사 텍스트 마이닝과 네트워크 분석을 통한 폭염의 사회·경제적 영향 유형 도출: 2012~2016년 사례)

  • Jung, Jae In;Lee, Kyoungjun;Kim, Seungbum
    • Atmosphere
    • /
    • v.30 no.3
    • /
    • pp.237-248
    • /
    • 2020
  • In order to effectively prepare for damage caused by weather events, it is important to proactively identify the possible impacts of weather phenomena on the domestic society and economy. Text mining and Network analysis are used in this paper to build a database of damage types and levels caused by heat wave. We collect news articles about heat wave from the SBS news website and determine the primary and secondary effects of that through network analysis. In addition to that, based on the frequency with which each impact keyword is mentioned, we estimate how much influence each factor has. As a result, the types of impacts caused by heat wave are efficiently derived. Among these types of impacts, we find that people in South Korea are mainly interested in algae and heat-related illness. Since this technique of analysis can be applied not only to news articles but also to social media contents, such as Twitter and Facebook, it is expected to be used as a useful tool for building weather impact databases.

Comparison and Analysis of Domestic and Foreign Sports Brands Using Text Mining and Opinion Mining Analysis (텍스트 마이닝과 오피니언 마이닝 분석을 활용한 국내외 스포츠용품 브랜드 비교·분석 연구)

  • Kim, Jae-Hwan;Lee, Jae-Moon
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.6
    • /
    • pp.217-234
    • /
    • 2018
  • In this study, big data analysis was conducted for domestic and international sports goods brands. Text Mining, TF-IDF, Opinion Mining, interestity graph were conducted through the social matrix program Textom and the fashion data analysis platform MISP. In order to examine the recent recognition of sports brands, the period of study is limited to 1 year from January 1, 2017 to December 31, 2017. As a result of analysis, first, we could confirm the products representing each brand. Second, I could confirm the marketing that represents each brand. Third, the common words extracted from each brand were identified. Fourth, the emotions of positive and negative of each brand were confirmed.