• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.14 no.1
• /
• pp.29-36
• /
• 2021

The amount of data generated has been growing exponentially, and the complexity of data has been increasing owing to the advancement of information technology (IT). Big data analysts and engineers have therefore been actively conducting research to minimize the analysis targets for faster processing and analysis of big data. Hadoop, which is widely used as a big data platform, provides various processing and analysis functions, including minimization of analysis targets through Hive, which is a subproject of Hadoop. However, Hive uses a vast amount of memory for data deduplication because it is implemented without considering the complexity of data. Therefore, an efficient algorithm has been proposed for data deduplication of complex structures. The performance evaluation results demonstrated that the proposed algorithm reduces the memory usage and data deduplication time by approximately 79% and 0.677%, respectively, compared to Hive. In the future, performance evaluation based on a large number of data nodes is required for a realistic verification of the proposed algorithm.

• The Journal of Bigdata
• /
• v.4 no.1
• /
• pp.129-142
• /
• 2019

Construction projects have experienced many failures due to incomplete production environments. Thus, the purpose of this study is to use ICT resources leased during the construction period at the construction site and to introduce the Mobile Cloud Computing Service, which utilizes Cloud Computing Service and mobile devices such as smart phones, tablet PCs, and notebooks instead of physically wired communication networks. The characteristics of Mobile Cloud, such as rapid accuracy, shared collaboration, and ubiquity, will affect the social network among various construction site participants. we conducted empirical research on the introduction of Mobile Cloud to promote information exchange and convergence among the participants and mutual trust, ultimately improving the project performance.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.21 no.1
• /
• pp.79-86
• /
• 2021

In order to increase the probability of success in big data projects, quantified techniques are required to analyze the root cause of risks from complex causes and establish optimal countermeasures. To this end, this study measures risk factors and relationships through SNA analysis and presents a way to respond to risks based on them. In other words, it derives a dependency network matrix by utilizing the results of correlation analysis between risk groups in the big data projects presented in the preliminary study and performs SNA analysis. In order to derive the dependency network matrix, partial correlation is obtained from the correlation between the risk nodes, and activity dependencies are derived by node by calculating the correlation influence and correlation dependency, thereby producing the causal relationship between the risk nodes and the degree of influence between all nodes in correlation. Recognizing the root cause of risks from networks between risk factors derived through SNA between risk factors enables more optimized and efficient risk management. This study is the first to apply SNA analysis techniques in relation to risk management response, and the results of this study are significant in that it not only optimizes the sequence of risk management for major risks in relation to risk management in IT projects but also presents a new risk analysis technique for risk control.

• Journal of the Society of Disaster Information
• /
• v.17 no.2
• /
• pp.254-265
• /
• 2021

Purpose: Inspection and diagnosis on the performance and safety through domestic port facilities have been conducted for over 20 years. However, the long-term development strategies and directions for facility renewal and performance improvement using the diagnosis history and results are not working in realistically. In particular, in the case of port structures with a long service life, there are many problems in terms of safety and functionality due to increasing of the large-sized ships, of port use frequency, and the effects of natural disasters due to climate change. Method: In this study, the maintenance history data of the gravity type quay in element level were collected, defined as big data, and a predictive approximation model was derived to estimate the pattern of deterioration and aging of the facility of project level based on the data. In particular, we compared and proposed models suitable for the use of big data by examining the validity of the state-based deterioration pattern and deterioration approximation model generated through machine learning algorithms of GP and SGP techniques. Result: As a result of reviewing the suitability of the proposed technique, it was considered that the RMSE and R² in GP technique were 0.9854 and 0.0721, and the SGP technique was 0.7246 and 0.2518. Conclusion: This research through machine learning techniques is expected to play an important role in decision-making on investment in port facilities in the future if port facility data collection is continuously performed in the future.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2017.11a
• /
• pp.652-654
• /
• 2017

본 프로젝트의 목적은 소상공인들을 위한 상권 분석, 트렌드 분석, 창업 지원 정책 소개, 커뮤니티 등을 제공하는 빅 데이터 기반의 웹 서비스를 구축하는 것이다. 일반적인 창업 관련 사이트는 정형데이터를 DB(Data Base)에 저장 후 관리되는 시스템으로, 이는 사용자 개개인에 맞는 맞춤형 정보를 제공하기 힘들다. 따라서 본 논문에서는 실시간 검색어 수집 및 분석을 통해 소상공인들이 창업을 희망할 때, 사용자에 맞는 정보를 제공해주는 맞춤형 서비스 연구에 대한 내용이다.

• The Journal of Bigdata
• /
• v.8 no.1
• /
• pp.83-97
• /
• 2023

Project evaluation is the process of evaluating the progress and results of a project. Smart city projects can be divided into system components (infrastructure, services, platforms), or projects can run simultaneously for multiple services. In addition, services are developed and expanded through additional projects. In order to ensure that the smart city, which is composed of various projects, proceeds in accordance with the goals and strategies, periodic project evaluation is required during the project implementation process. The smart city project evaluation system proposed in this paper is designed to provide comprehensive and objective indicators by reflecting various factors that must be considered for projects occurring in all stages of planning, design, construction, and operation of smart cities. The indicators derived from the evaluation system can be used by decision makers to determine the direction of smart city project development. In addition, it is designed so that the performance of the project can be evaluated interim before the end of the project and the feedback obtained from it can be reflected. To introduce the application method of the smart city project evaluation system proposed in this study, the evaluation system developed in this study was applied to the smart city project case of Incheon Free Economic Zone (IFEZ). Based on the evaluation results, items that can maximize the improvement effect of each smart city project item were presented, and the direction of smart city project implementation was suggested. By utilizing a smart city project evaluation system that reflects the characteristics of smart city projects that are composed of multiple projects, comprehensive planning and management of smart city projects will be possible, and this study will serve as a reference for identifying priority improvement factors for projects.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2014.04a
• /
• pp.700-703
• /
• 2014

최근 이슈가 되고 있는 하둡(hadoop) 패키지에 접목하여 많은 프로젝트들이 생겨나고 있으며, 이들 중 주요하게 떠오르고 있는 분야가 접근 제어 기술이다. 특히, 인터넷의 발전과 스마트 기기 사용자가 늘어남에 따라 데이터의 양이 증가하여, 데이터의 소유자와 사용자의 필요에 의한 접근 제어 기술이 필요하게 되었다. 본 논문에서는 접근 제어 기술의 필요성을 기반으로 HDFS(Hadoop Distributed File System, 하둡 분산 파일 시스템) 기반의 새로운 데이터 접근 제어 프레임워크를 제안한다. 제안하는 방법은 새로운 메타데이터 저장 모듈과 접근 관리 모듈을 만들어 데이터 접근 제어프레임워크를 구성함으로써, 빅데이터 플랫폼을 사용하는 사용자들을 위한 접근 제어 기능을 제공한다. 제안한 프레임워크는 기존 플랫폼에 추가적인 설치가 필요 없도록 하둡 내부에 설계하여 향후 활용도가 높을 것이라 기대된다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2013.11a
• /
• pp.176-179
• /
• 2013

본 연구는 최근 활성화된 스마트폰 내비게이션의 교통정보를 이용 시 상대적으로 품질이 취약한 지방지역의 빠른길 및 통행시간 정확도를 개선하는 방안에 대한 연구이다. 본 연구에서는 Hadoop기술을 이용하여 교통데이터에 대한 방대한 양의 데이터를 분석 처리한다. 특히, 실시간 교통정보가 제대로 수집되지 않는 지역의 도로 속도 정보에 대한 개선 방안으로 고객 단말로부터 올라오는 위치 데이터를 실시간으로 수집하여 전자지도 기본 속성값을 실시간 통계 데이터로 교체 반영하여 상대적으로 취약한 지방지역의 경로 품질 및 경로 탐색을 개선하는 방안을 제시하였다. 단말로부터 올라오는 많은 양의 GPS 위치정보등 대용량 데이터 처리를 위해 오픈소스프로젝트인 Hadoop플랫폼 환경에서 빅데이터 처리용 오픈소스를 활용하여 고가의 RDBMS를 대체하는 효과와 시간 단축의 효과를 기대할 수 있게 되었다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2020.05a
• /
• pp.60-62
• /
• 2020

데이터의 양은 기술의 발전으로 발생하는 크게 증가하였고 다양한 빅데이터 처리 플랫폼이 등장하고 있다. 이 중 가장 널리 사용되고 있는 품랫폼이 Apache 소프트웨어 재단에서 개발한 Hadoop이며, Hadoop은 IoT 분야에도 사용된다. 그러나 기존에 Hadoop 기반 IoT 센서 데이터 수집 분석 환경은 Hadoop의 코어 프로젝트인 HDFS의 Small File로 인한 네임노드의 과부하 문제와 Import된 데이터의 Update나 Delete가 불가능하다는 문제가 있다. 본 논문에서는 Apache Kudu와 Impala를 활용해 Lambda Architecture를 설계한다. 제안하는 Architecture는 IoT 센서 데이터를 Cold-Data와 Hot-Data로 분류해 각 성격에 맞는 스토리지에 저장하고 Batch를 동해 생성된 Batch-View와 Apache Kudu와 Impala를 통해 생성된 Real-time View를 활용해 기존 Hadoop 기반 IoT 센서 데이터 수집 분석 환경의 문제를 해결하고 사용자가 분석된 데이터에 접근하는 시간을 단축한다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.05a
• /
• pp.379-380
• /
• 2022

KAIST 캠퍼스 기반의 실습환경 구축을 위하여 캠퍼스 전체를 스마트시티 테스트베드로 사용하며 CCTV 네트워크 기반 모니터링/관제 시스템 구축, 교통, 방범, 가로등, CCTV, 교내 버스 등 인프라 통합 관제 및 보안 실습실 구축하고 교내 자율주행 기술 연구진과 실습 협력 추진을 통한 캠퍼스 기반의 실전 스마트 환경을 토대로 다각도의 보안 공격/방어 실습을 진행하고 지자체 및 컨소시엄 기업들과 산학협력 프로젝트를 진행하기 위하여 구축한 내용을 설명한다.