• Title/Summary/Keyword: Big-Data Platform

Search Result 503, Processing Time 0.031 seconds

Detection of Abnormal Ship Operation using a Big Data Platform based on Hadoop and Spark (하둡 및 스파크 기반 빅데이터 플랫폼을 이용한 선박 운항 효율 이상 상태 분석)

  • Lee, Taehyeon;Yu, Eun-seop;Park, Kaemyoung;Yu, Seongsang;Park, Jinpyo;Mun, Duhwan
    • Journal of the Korean Society of Manufacturing Process Engineers
    • /
    • v.18 no.6
    • /
    • pp.82-90
    • /
    • 2019
  • To reduce emissions of marine pollutants, regulations are being tightened around the world. In the shipbuilding and shipping industries, various countermeasures are being put forward. As there are limits to applying countermeasures to ships already in operation, however, it is necessary for these vessels to use energy efficiently. The sensors installed on ships typically gather a very large amount of data, and thus a big data platform is needed to manage and analyze the data. In this paper, we build a big data analysis platform based on Hadoop and Spark, and we present a method to detect abnormal ship operation using the platform. We also utilize real ship operation data to discuss the data analysis experiment.

A Study on the Platform for Big Data Analysis of Manufacturing Process (제조 공정 빅데이터 분석을 위한 플랫폼 연구)

  • Ku, Jin-Hee
    • Journal of Convergence for Information Technology
    • /
    • v.7 no.5
    • /
    • pp.177-182
    • /
    • 2017
  • As major ICT technologies such as IoT, cloud computing, and Big Data are being applied to manufacturing, smart factories are beginning to be built. The key of smart factory implementation is the ability to acquire and analyze data of the factory. Therefore, the need for a big data analysis platform is increasing. The purpose of this study is to construct a platform for big data analysis of manufacturing process and propose integrated method for analysis. The proposed platform is a RHadoop-based structure that integrates analysis tool R and Hadoop to distribute a large amount of datasets. It can store and analyze big data collected in the unit process and factory in the automation system directly in HBase, and it has overcome the limitations of RDB - based analysis. Such a platform should be developed in consideration of the unit process suitability for smart factories, and it is expected to be a guide to building IoT platforms for SMEs that intend to introduce smart factories into the manufacturing process.

A Construction Plan of Media Big Data Platform Through the Smart Media Meta-data Utilization (스마트 미디어 메타데이터 활용을 통한 미디어 빅데이터 플랫폼 구축 방안)

  • Hong, Jin-Woo
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.05a
    • /
    • pp.646-649
    • /
    • 2022
  • Media is a means for exchanging one's emotions or objective information in human society. Due to the spread of smart media due to the advent of digital media, media in modern society is not a simple means but encompasses the entire society in which humans live. It even took over the control function. Therefore, by examining the types and characteristics of media according to the service platform in which smart media is used, and building a big data hub for media based on this, new development and research directions for digital media can be explored. In this paper, we analyze various metadata generated through smart media and propose a plan for building a big data platform using it.

  • PDF

A Study on the Application of SE Approach to the Design of Health Monitoring Pilot Platform utilizing Big Data in the Nuclear Power Plant (NPP) (원전 상태 감시 및 조기 경보용 빅데이터 시범 플랫폼의 설계를 위한 시스템 엔지니어링 방법론 적용 연구)

  • Cha, Jae-Min;Shin, Junguk;Son, Choong-Yeon;Hwang, Dong-Sik;Yeom, Choong Sub
    • Journal of the Korean Society of Systems Engineering
    • /
    • v.11 no.2
    • /
    • pp.13-29
    • /
    • 2015
  • With the era of big data, the big data has been expected to have a large impact in the NPP safety areas. Although high interests of the big data for the NPP safety, only a limited researches concerning this issue are revealed. Especially, researches on the logical/physical structure and systematic design methods for the big data platform for the NPP safety were not dealt with. In this research, we design a new big data pilot platform for the NPP safety especially focusing on health monitoring and early warning services. For this, we propose a tailored design process based on SE approaches to manage inherent high complexities of the platform design. The proposed design process is consist of several steps from elicitate stakeholders to integration test via define operational concept and scenarios, and system requirements, design a conceptual functional architecture, select alternative physical modules for the derived functions and assess the applicability of the alternative modules, design a conceptual physical architecture, implement and integrate the physical modules. From the design process, this paper covers until the conceptual physical architecture design. In the following paper, the rest of the design process and results of the field test will be shown.

Shared Distributed Big-Data Processing Platform Model: a Study (대용량 분산처리 플랫폼 공유 모델 연구)

  • Jeong, Hwanjin;Kang, Taeho;Kim, GyuSeok;Shin, YoungHo;Jeong, Jinkyu
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.11
    • /
    • pp.601-613
    • /
    • 2016
  • With the increasing need for big data processing, building a shared big data processing platform is important to minimize time and monetary costs. In shared big data processing, multitenancy is a major requirement that needs to be addressed, in order to provide a single isolated personal big data platform for each user, but to share the underlying hardware is shared among users to increase hardware utilization. In this paper, we explore two well-known shared big data processing platform models. One is to use a native Hadoop cluster, and the other is to build a virtual Hadoop cluster for each user. For each model we verified whether it is sufficient to support multi-tenancy. We also present a method to complement unsupported multi-tenancy features in a native Hadoop cluster model. Lastly we built prototype platforms and compared the performance of both models.

A Study of Bigdata Platform for Supporting Engineering Services (엔지니어링 서비스 지원을 위한 클라우드 기반 빅데이터 플랫폼 개발 연구)

  • Seo, Dongwoo;Kim, Myungil;Park, Sangjin;Kim, Jaesung;Jeong, Seok Chan
    • The Journal of Bigdata
    • /
    • v.4 no.1
    • /
    • pp.119-127
    • /
    • 2019
  • This study explains how to solve engineering problems easily and efficiently by using cloud based big data platform. To do this, we propose a cloud based big data analysis platform. The application helps users easily create models for data analysis using cloud based big data analysis platform. Analytical models modeled using components are analyzed through an analysis engine. Our platform include pre-processing, analysis, and visualization algorithms required for data analysis. Finally, we show an application of effluent concentration in a sewage treatment process.

  • PDF

A Study on User Perception of Tourism Platform Using Big Data

  • Se-won Jeon;Sung-Woo Park;Youn Ju Ahn;Gi-Hwan Ryu
    • International journal of advanced smart convergence
    • /
    • v.13 no.1
    • /
    • pp.108-113
    • /
    • 2024
  • The purpose of this study is to analyze user perceptions of tourism platforms through big data. Data were collected from Naver, Daum, and Google as big data analysis channels. Using semantic network analysis with the keyword 'tourism platform,' a total of 29,265 words were collected. The collection period was set for two years, from August 31, 2021, to August 31, 2023. Keywords were analyzed for connected networks using TexTom and Ucinet programs for social network analysis. Keywords perceived by tourism platform users include 'travel,' 'diverse,' 'online,' 'service,' 'tourists,' 'reservation,' 'provision,' and 'region.' CONCOR analysis revealed four groups: 'platform information,' 'tourism information and products,' 'activation strategies for tourism platforms,' and 'tourism destination market.' This study aims to expand and activate services that meet the needs and preferences of users in the tourism field, as well as platforms tailored to the changing market, based on user perception, current status, and trend data on tourism platforms.

Analysis of Encryption Algorithm Performance by Workload in BigData Platform (빅데이터 플랫폼 환경에서의 워크로드별 암호화 알고리즘 성능 분석)

  • Lee, Sunju;Hur, Junbeom
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.29 no.6
    • /
    • pp.1305-1317
    • /
    • 2019
  • Although encryption for data protection is essential in the big data platform environment of public institutions and corporations, much performance verification studies on encryption algorithms considering actual big data workloads have not been conducted. In this paper, we analyzed the performance change of AES, ARIA, and 3DES for each of six workloads of big data by adding data and nodes in MongoDB environment. This enables us to identify the optimal block-based cryptographic algorithm for each workload in the big data platform environment, and test the performance of MongoDB by testing various workloads in data and node configurations using the NoSQL Database Benchmark (YCSB). We propose an optimized architecture that takes into account.

Big Data Analysis Platform Technology R&D Trend through Patent Analysis (특허분석을 통한 빅데이터 분석 플랫폼 기술 개발 동향)

  • Rho, Seungmin
    • Journal of Digital Convergence
    • /
    • v.12 no.9
    • /
    • pp.169-175
    • /
    • 2014
  • The ICT (information and communication technology) paradigm shift, including the burgeoning use of mobile, SNS, and smart devices, has resulted in an explosion of data along with lifestyle changes. We have thus arrived at the age of big data. In the meantime, a number of difficulties have arisen in terms of cost or on the technical side with respect to the use of large quantities of data. However, big data has begun to receive attention with the advent of efficient big data technologies such as Hadoop. In this paper, we discuss the patent analysis of big data platform technology research and development in major countries. Especially, we analyzed 2,568 patent applications and registered patents in four countries on December 2010.

Design of a Platform for Collecting and Analyzing Agricultural Big Data (농업 빅데이터 수집 및 분석을 위한 플랫폼 설계)

  • Nguyen, Van-Quyet;Nguyen, Sinh Ngoc;Kim, Kyungbaek
    • Journal of Digital Contents Society
    • /
    • v.18 no.1
    • /
    • pp.149-158
    • /
    • 2017
  • Big data have been presenting us with exciting opportunities and challenges in economic development. For instance, in the agriculture sector, mixing up of various agricultural data (e.g., weather data, soil data, etc.), and subsequently analyzing these data deliver valuable and helpful information to farmers and agribusinesses. However, massive data in agriculture are generated in every minute through multiple kinds of devices and services such as sensors and agricultural web markets. It leads to the challenges of big data problem including data collection, data storage, and data analysis. Although some systems have been proposed to address this problem, they are still restricted either in the type of data, the type of storage, or the size of data they can handle. In this paper, we propose a novel design of a platform for collecting and analyzing agricultural big data. The proposed platform supports (1) multiple methods of collecting data from various data sources using Flume and MapReduce; (2) multiple choices of data storage including HDFS, HBase, and Hive; and (3) big data analysis modules with Spark and Hadoop.