• Journal of Intelligence and Information Systems
• /
• v.23 no.3
• /
• pp.69-94
• /
• 2017

Recently, increase of demand for big data analysis has been driving the vigorous development of related technologies and tools. In addition, development of IT and increased penetration rate of smart devices are producing a large amount of data. According to this phenomenon, data analysis technology is rapidly becoming popular. Also, attempts to acquire insights through data analysis have been continuously increasing. It means that the big data analysis will be more important in various industries for the foreseeable future. Big data analysis is generally performed by a small number of experts and delivered to each demander of analysis. However, increase of interest about big data analysis arouses activation of computer programming education and development of many programs for data analysis. Accordingly, the entry barriers of big data analysis are gradually lowering and data analysis technology being spread out. As the result, big data analysis is expected to be performed by demanders of analysis themselves. Along with this, interest about various unstructured data is continually increasing. Especially, a lot of attention is focused on using text data. Emergence of new platforms and techniques using the web bring about mass production of text data and active attempt to analyze text data. Furthermore, result of text analysis has been utilized in various fields. Text mining is a concept that embraces various theories and techniques for text analysis. Many text mining techniques are utilized in this field for various research purposes, topic modeling is one of the most widely used and studied. Topic modeling is a technique that extracts the major issues from a lot of documents, identifies the documents that correspond to each issue and provides identified documents as a cluster. It is evaluated as a very useful technique in that reflect the semantic elements of the document. Traditional topic modeling is based on the distribution of key terms across the entire document. Thus, it is essential to analyze the entire document at once to identify topic of each document. This condition causes a long time in analysis process when topic modeling is applied to a lot of documents. In addition, it has a scalability problem that is an exponential increase in the processing time with the increase of analysis objects. This problem is particularly noticeable when the documents are distributed across multiple systems or regions. To overcome these problems, divide and conquer approach can be applied to topic modeling. It means dividing a large number of documents into sub-units and deriving topics through repetition of topic modeling to each unit. This method can be used for topic modeling on a large number of documents with limited system resources, and can improve processing speed of topic modeling. It also can significantly reduce analysis time and cost through ability to analyze documents in each location or place without combining analysis object documents. However, despite many advantages, this method has two major problems. First, the relationship between local topics derived from each unit and global topics derived from entire document is unclear. It means that in each document, local topics can be identified, but global topics cannot be identified. Second, a method for measuring the accuracy of the proposed methodology should be established. That is to say, assuming that global topic is ideal answer, the difference in a local topic on a global topic needs to be measured. By those difficulties, the study in this method is not performed sufficiently, compare with other studies dealing with topic modeling. In this paper, we propose a topic modeling approach to solve the above two problems. First of all, we divide the entire document cluster(Global set) into sub-clusters(Local set), and generate the reduced entire document cluster(RGS, Reduced global set) that consist of delegated documents extracted from each local set. We try to solve the first problem by mapping RGS topics and local topics. Along with this, we verify the accuracy of the proposed methodology by detecting documents, whether to be discerned as the same topic at result of global and local set. Using 24,000 news articles, we conduct experiments to evaluate practical applicability of the proposed methodology. In addition, through additional experiment, we confirmed that the proposed methodology can provide similar results to the entire topic modeling. We also proposed a reasonable method for comparing the result of both methods.

• Journal of Digital Convergence
• /
• v.20 no.5
• /
• pp.463-476
• /
• 2022

After industrialization in the past, the automobile industry has continued to grow centered on internal combustion engines, but is facing a major change with the recent 4th industrial revolution. Most companies are preparing for the transition to electric vehicles and autonomous driving. Therefore, in this study, topic modeling was performed based on LDA algorithm by collecting 4,002 domestic papers and 68,372 overseas papers that contain keywords related to CASE (Connectivity, Autonomous, Sharing, Electrification), which represent future automobile trends. As a result of the analysis, it was found that domestic research mainly focuses on macroscopic aspects such as traffic infrastructure, urban traffic efficiency, and traffic policy. Through this, the government's technical support for MaaS (Mobility-as-a-Service) is required in the domestic shared car sector, and the need for data opening by means of transportation was presented. It is judged that these analysis results can be used as basic data for the future automobile industry.

• Journal of Korea Multimedia Society
• /
• v.22 no.2
• /
• pp.250-259
• /
• 2019

Big data processing technology and artificial intelligence (AI) are increasingly attracting attention. Natural language processing is an important research area of artificial intelligence. In this paper, we use Korean news articles to extract topic distributions in documents and word distribution vectors in topics through LDA-based Topic Modeling. Then, we use Word2vec to vector words, and generate a weight matrix to derive the relevance SCORE considering the semantic relationship between the words. We propose a way to recommend documents in order of high score.

• Asia pacific journal of information systems
• /
• v.26 no.1
• /
• pp.66-79
• /
• 2016

The availability of detailed data on customers' online behaviors and advances in big data analysis techniques enable us to predict consumer behaviors. In the past, researchers have built purchase prediction models by analyzing clickstream data; however, these clickstream-based prediction models have had several limitations. In this study, we propose a new method for purchase prediction that combines information theory with machine learning techniques. Clickstreams from 5,000 panel members and data on their purchases of electronics, fashion, and cosmetics products were analyzed. Clickstreams were summarized using the 'entropy' concept from information theory, while 'random forests' method was applied to build prediction models. The results show that prediction accuracy of this new method ranges from 0.56 to 0.83, which is a significant improvement over values for clickstream-based prediction models presented in the past. The results indicate further that consumers' information search behaviors differ significantly across product categories.

• Journal of the Korea Convergence Society
• /
• v.11 no.8
• /
• pp.99-106
• /
• 2020

The purpose of this study is to find out whether image similarity between celebrities and brands on social network service be a determinant to predict advertising effectiveness. To this end, an advertising effect prediction model for celebrity endorsed advertising was created and its validity was verified through a machine learning method which is a big data analysis technique. Firstly, the celebrity-brand image similarity, which was used as an independent variable, was quantified by the association network theory with social big data, and secondly a multiple regression model which used data representing advertising effects as a dependent variable was repeatedly conducted to generate an advertising effect prediction model. The accuracy of the prediction model was decided by comparing the prediction results with the survey outcomes. As for a result, it was proved that the validity of the predictive modeling of advertising effects was secured since the classification accuracy of 75%, which is a criterion for judging validity, was shown. This study suggested a new methodological alternative and direction for big data-based modeling research through celebrity-brand image similarity structure based on social network theory, and effect prediction modeling by machine learning.

• Journal of Intelligence and Information Systems
• /
• v.20 no.2
• /
• pp.109-122
• /
• 2014

People are nowadays creating a tremendous amount of data on Social Network Service (SNS). In particular, the incorporation of SNS into mobile devices has resulted in massive amounts of data generation, thereby greatly influencing society. This is an unmatched phenomenon in history, and now we live in the Age of Big Data. SNS Data is defined as a condition of Big Data where the amount of data (volume), data input and output speeds (velocity), and the variety of data types (variety) are satisfied. If someone intends to discover the trend of an issue in SNS Big Data, this information can be used as a new important source for the creation of new values because this information covers the whole of society. In this study, a Twitter Issue Tracking System (TITS) is designed and established to meet the needs of analyzing SNS Big Data. TITS extracts issues from Twitter texts and visualizes them on the web. The proposed system provides the following four functions: (1) Provide the topic keyword set that corresponds to daily ranking; (2) Visualize the daily time series graph of a topic for the duration of a month; (3) Provide the importance of a topic through a treemap based on the score system and frequency; (4) Visualize the daily time-series graph of keywords by searching the keyword; The present study analyzes the Big Data generated by SNS in real time. SNS Big Data analysis requires various natural language processing techniques, including the removal of stop words, and noun extraction for processing various unrefined forms of unstructured data. In addition, such analysis requires the latest big data technology to process rapidly a large amount of real-time data, such as the Hadoop distributed system or NoSQL, which is an alternative to relational database. We built TITS based on Hadoop to optimize the processing of big data because Hadoop is designed to scale up from single node computing to thousands of machines. Furthermore, we use MongoDB, which is classified as a NoSQL database. In addition, MongoDB is an open source platform, document-oriented database that provides high performance, high availability, and automatic scaling. Unlike existing relational database, there are no schema or tables with MongoDB, and its most important goal is that of data accessibility and data processing performance. In the Age of Big Data, the visualization of Big Data is more attractive to the Big Data community because it helps analysts to examine such data easily and clearly. Therefore, TITS uses the d3.js library as a visualization tool. This library is designed for the purpose of creating Data Driven Documents that bind document object model (DOM) and any data; the interaction between data is easy and useful for managing real-time data stream with smooth animation. In addition, TITS uses a bootstrap made of pre-configured plug-in style sheets and JavaScript libraries to build a web system. The TITS Graphical User Interface (GUI) is designed using these libraries, and it is capable of detecting issues on Twitter in an easy and intuitive manner. The proposed work demonstrates the superiority of our issue detection techniques by matching detected issues with corresponding online news articles. The contributions of the present study are threefold. First, we suggest an alternative approach to real-time big data analysis, which has become an extremely important issue. Second, we apply a topic modeling technique that is used in various research areas, including Library and Information Science (LIS). Based on this, we can confirm the utility of storytelling and time series analysis. Third, we develop a web-based system, and make the system available for the real-time discovery of topics. The present study conducted experiments with nearly 150 million tweets in Korea during March 2013.

• The Journal of the Korea Contents Association
• /
• v.20 no.11
• /
• pp.28-37
• /
• 2020

This study was conducted to propose measures to improve crisis response to environmental issues by analyzing the news big data on the 'tap water larvae' situation and identifying related major keywords and topics. To accomplish this, 1,975 cases of 'tap water larvae' reported between July 13 to August 31, 2020 were divided into three periods and analyzed using topical modeling techniques. The analysis output 15 topics for each period. According to the result, the 'tap water larvae' incident, as reported in the media, is divided into the occurrence, diffusion, and rectification stages. The government's response and civilian risk consciousness and reaction could also be seen. Based on the result, the following measures to respond to environment risk is proposed. First, it is necessary to explore the various intertwined context with the 'tap water larvae' incident at its core and develop responsiveness to environmental problems through education which forms integrated views. Second, a role to monitor the environment must be implemented and civilian-participated environmental information must be shared through the application of internet communities. Third, the cultivation and deployment of environmental communicators who provide and communicate fast and accurate environment information is required. This study, as the first in Korea to use the topic modeling analysis method based on big data related to 'tap water larvae', has academic significance in that it has empirically and systematically analyzed environmental issues which appear as unstructured data. It also political significance as it suggests ways to improve environmental education and communication.

• Journal of People, Plants, and Environment
• /
• v.24 no.6
• /
• pp.563-572
• /
• 2021

Background and objective: The ongoing COVID-19 pandemic restricted daily life, forcing people to spend time indoors. With the growing interest in mental health issues and residential environments, 'pet plants' have been receiving attention during the unprecedented social distancing measures. This study aims to analyze the change in trends of pet plants before and during the COVID-19 pandemic and provide basic data for studies related to pet plants and directions of future development. Methods: A total of 2,016 news articles using the keyword 'pet plants' were collected on Naver News from January 1, 2018 to August 15, 2019 (609 articles) and January 1, 2020 to August 15, 2021 (1,407 articles). The texts were tokenized into words using KoNLPy package, ultimately coming up with 63,597 words. The analyses included frequency of keywords and topic modeling based on Latent Dirichlet Allocation (LDA) to identify the inherent meanings of related words and each topic. Results: Topic modeling generated three topics in each period (before and during the COVID-19), and the results showed that pet plants in daily life have become the object of 'emotional support' and 'healing' during social distancing. In particular, pet plants, which had been distributed as a solution to prevent solitary deaths and depression among seniors living alone, are now expanded to help resolve the social isolation of the general public suffering from COVID-19. The new term 'plant butler' became a new trend, and there was a change in the trend in which people shared their hobbies and information about pet plants and communicated with others in online. Conclusion: Based on these findings, the trend data of pet plants before and after the outbreak of COVID-19 can provide the basis for activating research on pet plants and setting the direction for development of related industries considering the continuous popularity and trend of indoor gardening and green hobby.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.8 no.9
• /
• pp.1313-1318
• /
• 2013

In order to actively respond to cyber attacks, not only the security systems such as IDS, IPS, and Firewalls, but also ESM, a system that detects cyber attacks by analyzing various log data, are preferably deployed. However, as the attacks be come more elaborate and advanced, existing signature-based detection methods start to face their limitations. In response to that, researches upon symptom detection technology based on attack modeling by employing big-data analysis technology are actively on-going. This symptom detection technology is effective when it can accurately extract features of attacks and manipulate them to successfully execute the attack modeling. We propose the ways to extract attack features which can play a role as the basis of the modeling and detect intelligent threats by carrying out scenario-based modeling.

• The Journal of Information Systems
• /
• v.28 no.4
• /
• pp.271-291
• /
• 2019

Purpose The purpose of this study is to propose a methodology to derive positive keywords and negative keywords through deep learning to classify reviews into positive reviews and negative ones, and then refine the results of topic modeling using these keywords. Design/methodology/approach In this study, we extracted topic keywords by performing LDA-based topic modeling. At the same time, we performed attention-based deep learning to identify positive and negative keywords. Finally, we refined the topic keywords using these keywords as filters. Findings We collected and analyzed about 6,000 English reviews of Gyeongbokgung, a representative tourist attraction in Korea, from Tripadvisor, a representative travel site. Experimental results show that the proposed methodology properly identifies positive and negative keywords describing major topics.