• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.9
• /
• pp.1199-1205
• /
• 2021

In the era of big data, not only structured data well organized in databases, but also the Internet, social network services, it is very important to effectively analyze unstructured big data such as web documents, e-mails, and social data generated in real time in mobile environment. Big data analysis is the process of creating new value by discovering meaningful new correlations, patterns, and trends in big data stored in data storage. We intend to summarize and visualize the analysis results through frequency analysis of unstructured article data using R language, a big data analysis tool. The data used in this study was analyzed for total 104 papers in the Mon-May 2021 among the journals of the Korea Institute of Information and Communication Engineering. In the final analysis results, the most frequently mentioned keyword was "Data", which ranked first 1,538 times. Therefore, based on the results of the analysis, the limitations of the study and theoretical implications are suggested.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2017.05a
• /
• pp.759-762
• /
• 2017

The proposed scheme is proposed secure transmission of fixed data and unstructured data among medical information transmitted in PACS. Unstructured data uses the AES encryption algorithm as sensitive data And transmitted using encrypted mosaic encryption techniques for the non-identification of medical images, which are regular data. In addition, we have experimented with increasing the key size for encryption. As a result, we did not notice any significant difference between 128 - bit size and 128 - key size even when encrypting the size of 196,256.

• The Korean Journal of Applied Statistics
• /
• v.27 no.2
• /
• pp.169-183
• /
• 2014

Massive and various types of data recorded everywhere are called big data. Therefore, it is important to analyze big data and to nd valuable information. Besides, to standardize unstructured big data is important for the application of statistical methods. In this paper, we will show how to standardize unstructured big data using MapReduce which is a distribution processing system. We also apply simple correspondence analysis and multiple correspondence analysis to nd the relationship and characteristic of direct relationship words for Samsung Electronics and The Korea Economic Daily newspaper as well as Apple Inc.

• Journal of the Korea Convergence Society
• /
• v.12 no.7
• /
• pp.45-51
• /
• 2021

In accordance with the rapid non-face-to-face environment and mobile first strategy, the explosive increase and creation of many structured/unstructured data every year demands new decision making and services using big data in all fields. However, there have been few reference cases of using the Hadoop Ecosystem, which uses the rapidly increasing big data every year to collect and load big data into a standard platform that can be applied in a practical environment, and then store and process well-established big data in a relational database. Therefore, in this study, after collecting unstructured data searched by keywords from social network services based on Hadoop 2.0 through three virtual machine servers in the Spring Framework environment, the collected unstructured data is loaded into Hadoop Distributed File System and HBase based on the loaded unstructured data, it was designed and implemented to store standardized big data in a relational database using a morpheme analyzer. In the future, research on clustering and classification and analysis using machine learning using Hive or Mahout for deep data analysis should be continued.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2019.05a
• /
• pp.121-121
• /
• 2019

특정 지역에 집중적으로 비가 내리는 현상인 국지성호우가 빈번히 발생함에 따라 하천 주변 사회기반시설의 침수 위험성이 증가하고 있다. 침수 위험성 판단 여부는 주로 수위정보를 이용하며 수위 예측은 대부분 수치모형을 이용한다. 본 연구에서는 빅데이터 기반의 RNN(Recurrent Neural Networks)기법 알고리즘을 활용하여 수위를 예측하였다. 연구대상지는 조위의 영향을 많이 받는 한강 전역을 대상으로 하였다. 2008년~2018년(10개년)의 실제 침수 피해 실적을 조사한 결과 잠수교, 한강대교, 청담대교 등에서 침수 피해 발생률이 높게 나타났고 SNS(Social Network Services)와 같은 비정형화 자료에서는 청담대교가 가장 많이 태그(Tag)되어 청담대교를 연구범위로 설정하였다. 본 연구에서는 Python에서 제공하는 Tensor flow Library를 이용하여 수위예측 알고리즘을 적용하였다. 데이터는 정형화 데이터와 비정형 데이터를 사용하였으며 정형화 데이터는 한강홍수 통제소나 기상청에서 제공하는 최근 10년간의 (2008~2018) 수위 및 강우량 자료를 수집하였다. 비정형화 데이터는 SNS를 이용하여 민간 정보를 수집하여 정형화된 자료와 함께 전체자료를 구축하였다. 민감도 분석을 통하여 모델의 은닉층(5), 학습률(0.02) 및 반복횟수(100)의 최적값을 설정하였고, 24시간 동안의 데이터를 이용하여 3시간 후의 수위를 예측하였다. 2008년~ 2017년 까지의 데이터는 학습 데이터로 사용하였으며 2018년의 수위를 예측 및 평가하였다. 2018년의 관측수위 자료와 비교한 결과 90% 이상의 데이터가 10% 이내의 오차를 나타내었으며, 첨두수위도 비교적 정확하게 예측되는 것을 확인하였다. 향후 수위와 강우량뿐만 아니라 다양한 인자들도 고려한다면 보다 신속하고 정확한 예측 정보를 얻을 수 있을 것으로 기대된다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2015.04a
• /
• pp.932-933
• /
• 2015

최근 데이터의 양이 급격하게 증가하면서 빅데이터의 시대가 도래했다. 빅데이터는 형식이 없는 비정형 데이터이므로 기존의 정형 데이터 처리 방법으로는 분석 및 데이터 처리가 불가능해졌다. 또한, 범죄예방에 대한 관심이 증가하면서, 범죄 데이터 분석의 수요가 증가하고 있다. 본 연구에서는 비정형 범죄 데이터를 분석, 예측 등의 전산처리를 하기 위한 정규화 메트릭을 설정하는 방안을 제안하고자 한다.

• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.11
• /
• pp.521-526
• /
• 2016

As on-line informal text data have massive in its volume and have unstructured characteristics in nature, there are limitations in applying traditional relational data model technologies for data storage and data analysis jobs. Moreover, using dynamically generating massive social data, social user's real-time reaction analysis tasks is hard to accomplish. In the paper, to capture easily the semantics of massive and informal on-line documents with unsupervised learning mechanism, we design and implement automatic topic extraction systems according to the mass of the words that consists a document. The input data set to the proposed system are generated first, using N-gram algorithm to build multiple words to capture the meaning of the sentences precisely, and Hadoop and Spark (In-memory distributed computing framework) are adopted to run topic model. In the experiment phases, TB level input data are processed for data preprocessing and proposed topic extraction steps are applied. We conclude that the proposed system shows good performance in extracting meaningful topics in time as the intermediate results come from main memories directly instead of an HDD reading.

• Smart Media Journal
• /
• v.11 no.9
• /
• pp.75-80
• /
• 2022

Recently, research and development on immersive virtual reality(VR) technology to provide a realistic experience is being widely conducted. To provide realistic experience in immersive virtual reality for VR participants, virtual environments should consist of high-realistic environments using 3D reconstruction. In this paper, to acquire 3D information in real space using multiple cameras in the reconstruction process, we propose a novel method of optimal camera placement for accurate reconstruction to minimize distortion of 3D information. Through our approach in this paper, real 3D information can obtain with minimized errors during environment reconstruction, and it is possible to provide a more immersive experience with the created virtual environment.

• Annual Conference on Human and Language Technology
• /
• 2018.10a
• /
• pp.487-492
• /
• 2018

4차산업 혁명의 여파로 국내에서는 다양한 분야에 인공지능과 빅데이터 기술을 활용하여 이전에 시행 중인 다양한 서비스 분야에 기술적 접목과 보완을 시도하고 있다. 특히 금융권에서 자금을 빌린 기업들을 대상으로 여신 안정성을 확보하고 선제적인 대응을 위해 온라인 뉴스기사들과 SNS 데이터 등을 이용하여 부실가능성을 예측하고 실제 업무에 도입하려는 시도들이 국내 주요 은행들을 중심으로 활발히 진행 중이다. 우리는 국내의 국책은행에서 수행한 비정형 데이터 기반의 기업의 부실징후 예측 시스템 개발 과정에서 시도된 다양한 분석 방법과 결과 그리고 과정 중에 발생한 문제점들에 관해 기술하고 관련 이슈들에 관하여 다룬다. 결과적으로 본 논문은 레이블이 없는 대량의 기사들에 레이블을 달기 위한 자동 태거(tagger) 개발과 뉴스 기사 예측 결과로부터 부실 가능성을 예측하기 위한 모델 및 성능 면에서 기사 예측 정확도 92%(AUC 0.96) 및 부실 가능성 기업 예측에서도 정형 데이터 분석결과에 견줄만한 성과를 이루었고 이에 관해 보고한다.

• Smart Media Journal
• /
• v.10 no.2
• /
• pp.16-21
• /
• 2021

Financial time series analysis plays a very important role economically and socially in modern society and is an important task affecting global development, but due to difficulties such as a lot of noise and uncertainty, financial time series analysis prediction is a difficult research topic. In this paper, we propose a market prediction method (MPIL) by converting unstructured data and structured data into images. For market prediction, it analyzes SNS and news data, which is unstructured data for n days, and converts the market data, which is structured data, to an image with the GADF algorithm, and predicts an ultra-short market that predicts the price of n+1 days through image learning. MPIL has an average accuracy of 56%, which is higher than the 50% average accuracy of the model that predicts the market with LSTM by using sentiment analysis used for existing market forecasting.