• Title/Summary/Keyword: 빅데이터플랫폼

Search Result 483, Processing Time 0.023 seconds

High-level Analytics Platform for Development of Distributed Deep Learning Model (분산 딥러닝 모델 개발을 위한 고수준 분석 플랫폼)

  • Park, Kyongseok;Yu, Chan Hee;Sarda, Komal;Um, Jung-Ho
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2020.11a
    • /
    • pp.804-806
    • /
    • 2020
  • 딥러닝(deep learning)은 기계학습 알고리즘 중 가장 널리 활용되고 있는 알고리즘이다. 딥러닝 기술은 산업, 과학, 국방 및 공공 부문을 비롯하여 거의 모든 분야에서 폭넓게 확산되고 있다. 그러나 기계학습 기술에 대한 이해와 프로그래밍 지식이 부족할 경우 자유롭게 활용하는 데는 제약이 따르고 있으며 빅데이터를 활용하여 일반 이용자들이 직접 분산 학습 모형을 개발하고 배포하는 데 어려움이 발생하고 있다. 이러한 요구를 충족시키기 위해 딥러닝 프레임워크의 저수준 API를 추상화하여 고수준 분석과 분산 딥러닝을 지원하고 일반 이용자들이 실무적으로 복잡한 딥러닝 기술을 활용할 수 있는 기술을 개발하였다. 플랫폼 개발과 함께 중요하게 고려해야 하는 요소 중 하나로 플랫폼의 배포와 확장성 역시 고려되어야 한다. 본 플랫폼은 조직 내 계산 자원을 이용하여 플랫폼을 배포할 수 있으며 상용 클라우드 서비스와 연동하여 배포할 수 있도록 설계됨에 따라 환경의 제약 없이 유연한 서비스 제공이 가능하다.

스마트 항로표지 수집정보의 연동 시험 시나리오 설계

  • 오세웅;김윤지;강동우;박세길;장준혁
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 2023.05a
    • /
    • pp.311-313
    • /
    • 2023
  • 자율운항선박, 스마트 해상물류 등 미래 해상환경의 패러다임 변화에 대응하여 항로표지 현장시설을 고도화하고 신해상교통인프라 지능화 및 정보서비스 개발을 위해 스마트 항로표지 및 연계기술 개발 사업을 수행하고 있다. 본 사업의 1단계 연구 성과로 스마트 항로표지 통합플랫폼과 항로표지 서비스 성능시험환경이 구축되는데, 본 연구에서는 스마트 항로표지 통합 플랫폼에 설치된 각종 센서에서 수집된 정보를 육상의 성능시험환경으로 연동 시험에 관한 시나리오를 설계하였다. 스마트 항로표지 통합 플랫폼 및 장착되는 센서에 사전에 설계된 해양자원명을 부여하고 항로표지 정보관리시스템의 등록하는 절차를 제시하였고, 실시간으로 수집되는 항로표지 정보를 연동하여 빅데이터 분석 플랫폼으로 저장하고, 저장한 정보를 항로표지 서비스로의 적용과 활용에 관한 시나리오 설계 결과를 검토하였다.

  • PDF

Comparison and Analysis of Domestic and Foreign Sports Brands Using Text Mining and Opinion Mining Analysis (텍스트 마이닝과 오피니언 마이닝 분석을 활용한 국내외 스포츠용품 브랜드 비교·분석 연구)

  • Kim, Jae-Hwan;Lee, Jae-Moon
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.6
    • /
    • pp.217-234
    • /
    • 2018
  • In this study, big data analysis was conducted for domestic and international sports goods brands. Text Mining, TF-IDF, Opinion Mining, interestity graph were conducted through the social matrix program Textom and the fashion data analysis platform MISP. In order to examine the recent recognition of sports brands, the period of study is limited to 1 year from January 1, 2017 to December 31, 2017. As a result of analysis, first, we could confirm the products representing each brand. Second, I could confirm the marketing that represents each brand. Third, the common words extracted from each brand were identified. Fourth, the emotions of positive and negative of each brand were confirmed.

Development of Data Profiling Software Supporting a Microservice Architecture (마이크로 서비스 아키텍처를 지원하는 데이터 프로파일링 소프트웨어의 개발)

  • Chang, Jae-Young;Kim, Jihoon;Jee, Seowoo
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.21 no.5
    • /
    • pp.127-134
    • /
    • 2021
  • Recently, acquisition of high quality data has become an important issue as the expansion of the big data industry. In order to acquiring high quality data, accurate evaluation of data quality should be preceded first. The quality of data can be evaluated through meta-information such as statistics on data, and the task to extract such meta-information is called data profiling. Until now, data profiling software has typically been provided as a component or an additional service of traditional data quality or visualization tools. Hence, it was not suitable for utilizing directly in various environments. To address this problem, this paper presents the development result of data profiling software based on a microservice architecture that can be serviced in various environments. The presented data profiler provides an easy-to-use interface that requests of meta-information can be serviced through the restful API. Also, a proposed data profiler is independent of a specific environment, thus can be integrated efficiently with the various big data platforms or data analysis tools.

A Study on the Security Threat Response in Smart Integrated Platforms (스마트 통합플랫폼 보안위협과 대응방안 연구)

  • Seung Jae Yoo
    • Convergence Security Journal
    • /
    • v.22 no.1
    • /
    • pp.129-134
    • /
    • 2022
  • A smart platform is defined as an evolved platform that realizes physical and virtual space into a hyper-connected environment by combining the existing platform and advanced IT technology. The hyper-connection that is the connection between information and information, infrastructure and infrastructure, infrastructure and information, or space and service, enables the realization and provision of high-quality services that significantly change the quality of life and environment of users. In addition, it is providing everyone with the effect of significantly improving the social safety net and personal health management level by implementing smart government and smart healthcare. A lot of information produced and consumed in these processes can act as a factor threatening the basic rights of the public and individuals by the informations themselves or through big data analysis. In particular, as the smart platform as a core function that forms the ecosystem of a smart city is naturally and continuously expanded, it faces a huge security burden in data processing and network operation. In this paper, platform components as core functions of smart city and appropriate security threats and countermeasures are studied.

A Study on the Analysis of Aviation Safety Data Structure and Standard Classification (항공안전데이터 구조 분석 및 표준 분류체계에 관한 연구)

  • Kim, Jun Hwan;Lim, Jae Jin;Lee, Jang Ryong
    • Journal of the Korean Society for Aviation and Aeronautics
    • /
    • v.28 no.4
    • /
    • pp.89-101
    • /
    • 2020
  • In order to enhance the safety of the international aviation industry, the International Civil Aviation Organization has recommended establishing an operational foundation for systematic and integrated collection, storage, analysis and sharing of aviation safety data. Accordingly, the Korea aviation industry also needs to comprehensively manage the safety data which generated and collected by various stakeholders related to aviation safety, and through this, it is necessary to previously identify and remove hazards that may cause accident. For more effective data management and utilization, a standard structure should be established to enable integrated management and sharing of safety data. Therefore, this study aims to propose the framework about how to manage and integrate the aviation safety data for big data-based aviation safety management and shared platform.

A Study on Wildfire Disaster Response based on Cases of International Disaster Safety Management Systems (해외 재난 안전관리 시스템 사례기반 산불재난대응 연구)

  • Lee, Jihyun;Park, Minsoo;Jung, Dae-kyo;Park, Seunghee
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.40 no.3
    • /
    • pp.345-352
    • /
    • 2020
  • Forest fires generate many types of risk as well as a wide and varied range of damage. Various studies and systems have emerged in response to wildfire disasters. International wildfire disaster safety management systems apply advanced technologies such as utilizing big data, GIS-based systems, and decision-making systems. This study analyzes South Korea's and other countries' forest fire disaster safety management systems, and suggests alternatives for wildfire disaster safety management in Korea. First, a means of integrating information, including field information, obtained by domestic agencies is proposed. Second, a method of applying big data to the disaster response system is proposed. Third, a decision-making system is applied to an existing GIS-based system. When applying the above countermeasures to Korea's existing disaster safety management system, various information and data can be visualized and thus more easily identified, leading to more effective decision-making and reduced fire damage.

A Study on the Improvement of Heat Energy Efficiency for Utilities of Heat Consumer Plants based on Reinforcement Learning (강화학습을 기반으로 하는 열사용자 기계실 설비의 열효율 향상에 대한 연구)

  • Kim, Young-Gon;Heo, Keol;You, Ga-Eun;Lim, Hyun-Seo;Choi, Jung-In;Ku, Ki-Dong;Eom, Jae-Sik;Jeon, Young-Shin
    • Journal of Energy Engineering
    • /
    • v.27 no.2
    • /
    • pp.26-31
    • /
    • 2018
  • This paper introduces a study to improve the thermal efficiency of the district heating user control facility based on reinforcement learning. As an example, it is proposed a general method of constructing a deep Q learning network(DQN) using deep Q learning, which is a reinforcement learning algorithm that does not specify a model. In addition, it is also introduced the big data platform system and the integrated heat management system which are specialized in energy field applied in processing huge amount of data processing from IoT sensor installed in many thermal energy control facilities.

Study on the shared in a distributed processing system (분산처리시스템에서 공유에 대한 고찰)

  • Kim, Gyu-Seok
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2016.10a
    • /
    • pp.346-347
    • /
    • 2016
  • 인프라로써의 전산자원은 사용자간에 공동 활용될 때 비용 효율을 얻을 수 있다. 이것은 web, was, dbms 중심의 기존 정보시스템 뿐 만 아니라 빅 데이터 등 분산처리가 요구되는 분야에서도 마찬가지이다. 분산처리 시스템에서 보안상 안전하게 전산 리소스를 공유하기 위한 요소를 살펴보고 사용자별로 격리된 클러스터를 제공하는 분산처리 공유 플랫폼을 구축하여 확인해 보았다.

Exploring Issues Related to the Metaverse from the Educational Perspective Using Text Mining Techniques - Focusing on News Big Data (텍스트마이닝 기법을 활용한 교육관점에서의 메타버스 관련 이슈 탐색 - 뉴스 빅데이터를 중심으로)

  • Park, Ju-Yeon;Jeong, Do-Heon
    • Journal of Industrial Convergence
    • /
    • v.20 no.6
    • /
    • pp.27-35
    • /
    • 2022
  • The purpose of this study is to analyze the metaverse-related issues in the news big data from an educational perspective, explore their characteristics, and provide implications for the educational applicability of the metaverse and future education. To this end, 41,366 cases of metaverse-related data searched on portal sites were collected, and weight values of all extracted keywords were calculated and ranked using TF-IDF, a representative term weight model, and then word cloud visualization analysis was performed. In addition, major topics were analyzed using topic modeling(LDA), a sophisticated probability-based text mining technique. As a result of the study, topics such as platform industry, future talent, and extension in technology were derived as core issues of the metaverse from an educational perspective. In addition, as a result of performing secondary data analysis under three key themes of technology, job, and education, it was found that metaverse has issues related to education platform innovation, future job innovation, and future competency innovation in future education. This study is meaningful in that it analyzes a vast amount of news big data in stages to draw issues from an education perspective and provide implications for future education.