• Title/Summary/Keyword: BigData Platform

Search Result 506, Processing Time 0.029 seconds

Analysis of Research Trends of Ecosystem Service Related to Climate Change Using Big-data (빅데이터를 활용한 기후변화와 연계된 생태계서비스 연구 동향분석)

  • Seo, Ja-Yoo;Choi, Yo-Han;Baek, Ji-Won;Kim, Su-Kyoung;Kim, Ho-Gul;Song, Won-Kyong;Joo, Woo-Yeong;Park, Chan
    • Journal of the Korean Society of Environmental Restoration Technology
    • /
    • v.24 no.6
    • /
    • pp.1-13
    • /
    • 2021
  • This study was performed to investigate the ecosystem service patterns in relation to climate change acceleration utilizing big data analysis. This study aimed to use big data analysis as one of the network of views to identify convergent thinking in two fields: climate change and ecosystem service. The keywords were analysed to ascertain if there were any differences in the perceiving problems, policy direction, climate change implications, and regional differences. In addition, we examined the research keywords of each continent, the centre of ecosystem service research, and the topics to be referred to in domestic research. The results of the analysis are as follows: First, the keyword centrality of climate change is similar to the detailed indicators of The Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) regulations, content, and non-material ecosystem services. Second, the cross-analysis of terms in two journals showed a difference in value-oriented point; the Ecosystem Service Journal identified green infrastructure as having economic value, whereas the Climate Change Journal perceives water, forest, carbon, and biodiversity as management topics. The Climate Change Journal, but not the former, focuses on future predictions. Third, the analysis of the research topics according to continents showed that water and soil are closely related to the economy, and thus, play an important role in policy formulation. This disparity is due to differences in each continent's environmental characteristics, as well as economic and policy issues. This fact can be used to refer to the direction of research on ecosystem services in Korea. Consistent with the recent trend of expanding research regarding the impacts of climate change, it is necessary to study strategies to scientifically predict and respond to the negative effects of climate change.

Lambda Architecture Used Apache Kudu and Impala (Apache Kudu와 Impala를 활용한 Lambda Architecture 설계)

  • Hwang, Yun-Young;Lee, Pil-Won;Shin, Yong-Tae
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.9
    • /
    • pp.207-212
    • /
    • 2020
  • The amount of data has increased significantly due to advances in technology, and various big data processing platforms are emerging, to handle it. Among them, the most widely used platform is Hadoop developed by the Apache Software Foundation, and Hadoop is also used in the IoT field. However, the existing Hadoop-based IoT sensor data collection and analysis environment has a problem of overloading the name node due to HDFS' Small File, which is Hadoop's core project, and it is impossible to update or delete the imported data. This paper uses Apache Kudu and Impala to design Lambda Architecture. The proposed Architecture classifies IoT sensor data into Cold-Data and Hot-Data, stores it in storage according to each personality, and uses Batch-View created through Batch and Real-time View generated through Apache Kudu and Impala to solve problems in the existing Hadoop-based IoT sensor data collection analysis environment and shorten the time users access to the analyzed data.

A Study on Environmental Factor Recommendation Technology based on Deep Learning for Digital Agriculture (디지털 농업을 위한 딥러닝 기반의 환경 인자 추천 기술 연구)

  • Han-Jin Cho
    • Smart Media Journal
    • /
    • v.12 no.5
    • /
    • pp.65-72
    • /
    • 2023
  • Smart Farm means creating new value in various fields related to agriculture, including not only agricultural production but also distribution and consumption through the convergence of agriculture and ICT. In Korea, a rental smart farm is created to spread smart agriculture, and a smart farm big data platform is established to promote data collection and utilization. It is pushing for digital transformation of agricultural products distribution from production areas to consumption areas, such as expanding smart APCs, operating online exchanges, and digitizing wholesale market transaction information. As such, although agricultural data is generated according to characteristics from various sources, it is only used as a service using statistics and standardized data. This is because there are limitations due to distributed data collection from agriculture to production, distribution, and consumption, and it is difficult to collect and process various types of data from various sources. Therefore, in this paper, we analyze the current state of domestic agricultural data collection and sharing for digital agriculture and propose a data collection and linkage method for artificial intelligence services. And, using the proposed data, we propose a deep learning-based environmental factor recommendation method.

Irregular Bigdata Analysis and Considerations for Civil Complaint Based on Design Thinking (비정형 빅데이터 분석 및 디자인씽킹을 활용한 민원문제 해결에 대한 고찰)

  • Kim, Tae-Hyung;Park, Byung-Jae;Suh, Eung-Kyo
    • The Journal of Industrial Distribution & Business
    • /
    • v.9 no.8
    • /
    • pp.51-60
    • /
    • 2018
  • Purpose - Civil affairs are increasing in various forms, but civil servants who are able to handle them want to reduce the complaints and provide keywords that will help in the future due to their lack of time. While various ideas are presented and implemented as policies in solving civil affairs, there are many cases that are not policies that people can sympathize with. Therefore, it is necessary to analyze the complaints accurately and to present correct solutions to the analyzed civil complaint data. Research design, data, and methodology - We analyzed the complaints data for the last three years and found out how to solve the problems of Yongin City and alleviate the burdens of civil servants. To do this, the Hadoop platform and Design Thinking process were reviewed, and proposed a new process to fuse it. The big data analysis stage focuses on civil complaints - Civil data extraction - Civil data analysis - Categorization of the year by keywords analyzing them and the needs of citizens were identified. In the forecast analysis for deriving insights, - The case of innovation case study - Idea derivation - Idea evaluation - Prototyping - Case analysis stage used. Results - Through this, a creative idea of providing free transportation cards to solve the major issues of construction, apartment, installation, and vehicle problems was discovered. There is a specific problem of how to provide these services to certain areas, but there is a pressing need for a policy that can contribute as much as it can to the citizens who are suffering from various problems at this moment. Conclusions - In the past, there were many cases in which free traffic cards were issued mainly to the elderly or disabled. In other countries, foreign residents of other area visit the areas for accommodation, and may give out free transportation cards as well. In this case, the local government will be able to set up a framework to present with a win-win scenario in various ways. It is necessary to reorganize the process in future studies so that the actual solution will be adopted, reduce civil complaints, help establish policies in the future, and be applied in other cities as well.

Distributed Edge Computing for DNA-Based Intelligent Services and Applications: A Review (딥러닝을 사용하는 IoT빅데이터 인프라에 필요한 DNA 기술을 위한 분산 엣지 컴퓨팅기술 리뷰)

  • Alemayehu, Temesgen Seyoum;Cho, We-Duke
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.12
    • /
    • pp.291-306
    • /
    • 2020
  • Nowadays, Data-Network-AI (DNA)-based intelligent services and applications have become a reality to provide a new dimension of services that improve the quality of life and productivity of businesses. Artificial intelligence (AI) can enhance the value of IoT data (data collected by IoT devices). The internet of things (IoT) promotes the learning and intelligence capability of AI. To extract insights from massive volume IoT data in real-time using deep learning, processing capability needs to happen in the IoT end devices where data is generated. However, deep learning requires a significant number of computational resources that may not be available at the IoT end devices. Such problems have been addressed by transporting bulks of data from the IoT end devices to the cloud datacenters for processing. But transferring IoT big data to the cloud incurs prohibitively high transmission delay and privacy issues which are a major concern. Edge computing, where distributed computing nodes are placed close to the IoT end devices, is a viable solution to meet the high computation and low-latency requirements and to preserve the privacy of users. This paper provides a comprehensive review of the current state of leveraging deep learning within edge computing to unleash the potential of IoT big data generated from IoT end devices. We believe that the revision will have a contribution to the development of DNA-based intelligent services and applications. It describes the different distributed training and inference architectures of deep learning models across multiple nodes of the edge computing platform. It also provides the different privacy-preserving approaches of deep learning on the edge computing environment and the various application domains where deep learning on the network edge can be useful. Finally, it discusses open issues and challenges leveraging deep learning within edge computing.

High-performance computing for SARS-CoV-2 RNAs clustering: a data science-based genomics approach

  • Oujja, Anas;Abid, Mohamed Riduan;Boumhidi, Jaouad;Bourhnane, Safae;Mourhir, Asmaa;Merchant, Fatima;Benhaddou, Driss
    • Genomics & Informatics
    • /
    • v.19 no.4
    • /
    • pp.49.1-49.11
    • /
    • 2021
  • Nowadays, Genomic data constitutes one of the fastest growing datasets in the world. As of 2025, it is supposed to become the fourth largest source of Big Data, and thus mandating adequate high-performance computing (HPC) platform for processing. With the latest unprecedented and unpredictable mutations in severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the research community is in crucial need for ICT tools to process SARS-CoV-2 RNA data, e.g., by classifying it (i.e., clustering) and thus assisting in tracking virus mutations and predict future ones. In this paper, we are presenting an HPC-based SARS-CoV-2 RNAs clustering tool. We are adopting a data science approach, from data collection, through analysis, to visualization. In the analysis step, we present how our clustering approach leverages on HPC and the longest common subsequence (LCS) algorithm. The approach uses the Hadoop MapReduce programming paradigm and adapts the LCS algorithm in order to efficiently compute the length of the LCS for each pair of SARS-CoV-2 RNA sequences. The latter are extracted from the U.S. National Center for Biotechnology Information (NCBI) Virus repository. The computed LCS lengths are used to measure the dissimilarities between RNA sequences in order to work out existing clusters. In addition to that, we present a comparative study of the LCS algorithm performance based on variable workloads and different numbers of Hadoop worker nodes.

The Challenges of Public Policy Management for the 4th Industrial Revolution (제4차산업혁명에 대응하는 공공관리의 변화와 향후 과제: 사회-기술시스템론적 접근을 중심으로)

  • Jin, Sang-Ki;Bang, Min-Seok
    • Journal of Digital Convergence
    • /
    • v.16 no.4
    • /
    • pp.39-47
    • /
    • 2018
  • This paper aim to propose and suggest for government to reinvent and re-steering the policy on the 4th industrial revolution in Korea. To do it, this paper annalize the advanced country's policies and hard-data sets from many international research institutes. So this paper emphasized the concept and frame of socio-technical system approach by Geels. According to this approach, this paper can get a conclusion that Korea government has to focus on the roles to solve the big social dilemma with high-tech tools from the 4th industrial revolution. And this paper point out the necessary of increasing the diversity and multidisciplinary in 4th industrial revolution policy of Korean government. These findings of this paper recommend for Korea to redesigns the governance mechanism and legislation platform and to design the future readiness.

An Exploratory Study of Psychological Characteristics of Metaverse Users (메타버스 이용자의 심리 특성 탐색 연구)

  • Hyeonjeong Kim;HyunJung Kim;Beomsoo Kim;Hwan-Ho Noh
    • Knowledge Management Research
    • /
    • v.24 no.4
    • /
    • pp.63-85
    • /
    • 2023
  • This study aims to identify the primary user group in the growing metaverse space based on the increased interest during the COVID-19 era. It also aims to explore the predictive factors for metaverse adoption. To predict online activities, the study examined user purposes, motivations, and relevant demographic factors as predictive variables through model analysis. The data from the Korean Media Panel Survey were used, and a two-stage analysis with the Heckman two-stage sample selection model was conducted to predict metaverse users. The analysis revealed that the key factors influencing metaverse adoption were offline activities, openness, OTT usage, and purchasing of paid content. Moreover, in the second stage model, openness, gender, and paid content purchases were identified as significant variables for increasing metaverse usage time. These results indicate that understanding metaverse users is essential in the context of the rising interest in online activities during the COVID-19 era and can provide valuable insights for metaverse platform-related companies and developers.

Development of Retargetable Hadoop Simulation Environment Based on DEVS Formalism (DEVS 형식론 기반의 재겨냥성 하둡 시뮬레이션 환경 개발)

  • Kim, Byeong Soo;Kang, Bong Gu;Kim, Tag Gon;Song, Hae Sang
    • Journal of the Korea Society for Simulation
    • /
    • v.26 no.4
    • /
    • pp.51-61
    • /
    • 2017
  • Hadoop platform is a representative storing and managing platform for big data. Hadoop consists of distributed computing system called MapReduce and distributed file system called HDFS. It is important to analyse the effectiveness according to the change of cluster constructions and several parameters. However, since it is hard to construct thousands of clusters and analyse the constructed system, simulation method is required to analyse the system. This paper proposes Hadoop simulator based on DEVS formalism which provides hierarchical and modular modeling. Hadoop simulator provides a retargetable experimental environment that is possible to change of various parameters, algorithms and models. It is also possible to design input models reflecting the characteristics of Hadoop applications. To maximize the user's convenience, the user interface, real-time model viewer, and input scenario editor are also provided. In this paper, we validate Hadoop Simulator through the comparison with the Hadoop execution results and perform various experiments.

Changes in the Industrial Structure caused by the IoT and AI (사물인터넷과 AI가 가져올 산업구조의 변화)

  • Kim, Jang-Hwan
    • Convergence Security Journal
    • /
    • v.17 no.5
    • /
    • pp.93-99
    • /
    • 2017
  • Recently IoT(Internet of Things) service industry has grown very rapidly. In this paper, we investigated the changes in IoT service industry as well as new direction of human life in future global society. Under these changing market conditions, competition has been also changed into global and ecological competition. But compared to the platform initiatives and ecological strategies of global companies, Korean companies' vision of building ecosystems is still unclear. In addition, there is a need of internetworking between mobile and IoT services. IoT security Protocol has weakness of leaking out information from Gateway which connected wire and wireless communication. As such, we investigate the structure of IoT and AI service ecosystem in order to gain strategic implications and insights for the security industry in this paper.