• Journal of Internet Computing and Services
• /
• v.15 no.1
• /
• pp.103-112
• /
• 2014

In recent years, the rapid diffusion of smart devices and growth of internet usage and social media has led to a constant production of huge amount of valuable data set that includes personal information, buying patterns, location information and other things. IT and Production Infrastructure has also started to produce its own data with the vitalization of M2M (Machine-to-Machine) and IoT (Internet of Things). This analysis study researches the applicable effects of Structured and Unstructured Big Data in various business circumstances, and purposes to find out the value creation method for a corporation through the Structured and Unstructured Big Data case studies. The result demonstrates that corporations looking for the optimized big data utilization plan could maximize their creative values by utilizing Unstructured and Structured Big Data generated interior and exterior of corporations.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.9
• /
• pp.1199-1205
• /
• 2021

In the era of big data, not only structured data well organized in databases, but also the Internet, social network services, it is very important to effectively analyze unstructured big data such as web documents, e-mails, and social data generated in real time in mobile environment. Big data analysis is the process of creating new value by discovering meaningful new correlations, patterns, and trends in big data stored in data storage. We intend to summarize and visualize the analysis results through frequency analysis of unstructured article data using R language, a big data analysis tool. The data used in this study was analyzed for total 104 papers in the Mon-May 2021 among the journals of the Korea Institute of Information and Communication Engineering. In the final analysis results, the most frequently mentioned keyword was "Data", which ranked first 1,538 times. Therefore, based on the results of the analysis, the limitations of the study and theoretical implications are suggested.

• The Journal of Bigdata
• /
• v.6 no.2
• /
• pp.169-179
• /
• 2021

Consumers' food consumption behavior is likely to be affected not only by structured data such as consumer panel data but also by unstructured data such as mass media and social media. In this study, a deep learning-based consumption prediction model is generated and verified for the fusion data set linking structured data and unstructured data related to food consumption. The results of the study showed that model accuracy was improved when combining structured data and unstructured data. In addition, unstructured data were found to improve model predictability. As a result of using the SHAP technique to identify the importance of variables, it was found that variables related to blog and video data were on the top list and had a positive correlation with the amount of paprika purchased. In addition, according to the experimental results, it was confirmed that the machine learning model showed higher accuracy than the deep learning model and could be an efficient alternative to the existing time series analysis modeling.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2014.04a
• /
• pp.631-634
• /
• 2014

최근 스마트폰의 보급으로 블로그, SNS 등에서 방대한 양의 데이터가 발생함에 따라 이를 수집하고 분석하는 작업의 중요성이 커지고 있다. 이러한 데이터는 크게 정형 데이터와 비정형 데이터로 나눌 수 있는데, 특히 비정형 데이터는 전체 데이터의 약 80%를 차지할 정도로 그 양과 가치가 매우 크다. 이 논문에서는 빅데이터 환경에서 발생하는 이러한 비정형 데이터를 수집하는 모듈 중 가장 널리 알려진 Chukwa와 Flume에 대한 개발 및 비교 분석을 시도 하였다.

• Journal of the Korea Convergence Society
• /
• v.12 no.7
• /
• pp.45-51
• /
• 2021

In accordance with the rapid non-face-to-face environment and mobile first strategy, the explosive increase and creation of many structured/unstructured data every year demands new decision making and services using big data in all fields. However, there have been few reference cases of using the Hadoop Ecosystem, which uses the rapidly increasing big data every year to collect and load big data into a standard platform that can be applied in a practical environment, and then store and process well-established big data in a relational database. Therefore, in this study, after collecting unstructured data searched by keywords from social network services based on Hadoop 2.0 through three virtual machine servers in the Spring Framework environment, the collected unstructured data is loaded into Hadoop Distributed File System and HBase based on the loaded unstructured data, it was designed and implemented to store standardized big data in a relational database using a morpheme analyzer. In the future, research on clustering and classification and analysis using machine learning using Hive or Mahout for deep data analysis should be continued.

• Convergence Security Journal
• /
• v.22 no.4
• /
• pp.85-97
• /
• 2022

With the development of big data technology, data is rapidly entering a hyperconnected intelligent society that accelerates innovative growth in all industries. The convergence industry, which holds and utilizes various high-quality data, is becoming a new growth engine, and big data is fused to various traditional industries. In particular, in the medical field, structured data such as electronic medical record data and unstructured medical data such as CT and MRI are used together to increase the accuracy of disease prediction and diagnosis. Currently, the importance and size of unstructured data are increasing day by day in the medical industry, but conventional data security technologies and policies are structured data-oriented, and considerations for the security and utilization of unstructured data are insufficient. In order for medical treatment using big data to be activated in the future, data diversity and security must be internalized and organically linked at the stage of data construction, distribution, and utilization. In this paper, the current status of domestic and foreign data security systems and technologies is analyzed. After that, it is proposed to add unstructured data-centered de-identification technology to the guidelines for unstructured data and technology application cases in the industry so that unstructured data can be actively used in the medical field, and to establish standards for judging personal information for unstructured data. Furthermore, an object feature-based identification ID that can be used for unstructured data without infringing on personal information is proposed.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2013.05a
• /
• pp.904-907
• /
• 2013

최근 정보통신기술이 하루가 다르게 발전함에 따라 하루에도 수많은 데이터가 흘러나오는 최근의 추세이다. 정형 데이터 뿐 아니라 비정형 데이터 분석까지 진행하는 최근의 추세에 맞춰 현 빅데이터 기술 동향을 분석한다. 빅데이터 시대를 맞아 기존의 데이터웨어하우스(DW)와 발전된 데이터웨어하우스(DW) 어플라이언스에 대해 분석하고 향후 발전 전망과 방향을 제시한다.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2020.06a
• /
• pp.388-388
• /
• 2020

매년 국지성호우 및 태풍으로 인해 하천 범람이나 저지대침수가 발생하고 있으며 이는 인명 피해 사례로 이어지기도 한다. 피해 발생을 최소화시키기 위해 강우와 유량과 같은 정형데이터로 홍수예보가 이뤄지고 있으나 기존의 정형데이터만 사용하다보니 도심지역이나 소규모 하천에서 인명 피해 예측에 어려움이 있다. 이를 보완하기 위해서는 인구의 유동성을 고려한 비정형데이터를 활용해야 한다. 최근 소셜 네트워크 서비스(SNS)의 사용자가 증가됨에 따라 텍스트나 사진과 같은 다양한 비정형데이터가 생성되고 있다. 이렇게 생성된 데이터는 다양한 분야에서 활용되고 있으며 특히 지진이나 홍수와 같은 재난 발생 시 유용한 데이터로 활용된 사례가 증가하고 있다. 이는 사람들이 GIS와 같은 위치정보나 시간 등을 포함한 다양한 정보를 포함하기 때문이다. 하지만 이렇게 생산된 비정형데이터를 기존 물리적 기반의 수문모형의 데이터로 활용하기에는 많은 한계점이 있다. 따라서 본 연구에서는 SNS 채널을 통해 생성된 비정형 데이터들을 인공신경망모형에 적용하여 하천수위를 예측하였다.

• The Korean Journal of Applied Statistics
• /
• v.27 no.2
• /
• pp.169-183
• /
• 2014

Massive and various types of data recorded everywhere are called big data. Therefore, it is important to analyze big data and to nd valuable information. Besides, to standardize unstructured big data is important for the application of statistical methods. In this paper, we will show how to standardize unstructured big data using MapReduce which is a distribution processing system. We also apply simple correspondence analysis and multiple correspondence analysis to nd the relationship and characteristic of direct relationship words for Samsung Electronics and The Korea Economic Daily newspaper as well as Apple Inc.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2015.04a
• /
• pp.170-171
• /
• 2015

최근 디지털 정보량의 기하급수적인 증가에 따라 대규모 데이터인 빅데이터가 등장하였다. 빅데이터는 데이터가 실시간으로 매우 빠르게 생성되며 다양한 형태의 데이터를 가지며 이 데이터를 수집, 처리, 분석을 통해 새로운 지식을 창출한다. 그러나 기존의 ETL(Exact/Transform/Load) 연구에서 이러한 빅데이터를 처리 하는데 성능 저하가 발생되고 있으며 비정형 데이터를 관리할 수 없다. 따라서 본 논문에서는 기존의 ETL 처리의 한계를 극복하기 위해서 하둡을 이용하여 ETL 상에서 처리 속도를 높이고 비정형 데이터를 처리할 수 있는 빅데이터 처리 시스템을 제안하고자 한다.