• Title/Summary/Keyword: Cloud collection

Search Result 135, Processing Time 0.021 seconds

Implementation of AIoT Edge Cluster System via Distributed Deep Learning Pipeline

  • Jeon, Sung-Ho;Lee, Cheol-Gyu;Lee, Jae-Deok;Kim, Bo-Seok;Kim, Joo-Man
    • International journal of advanced smart convergence
    • /
    • v.10 no.4
    • /
    • pp.278-288
    • /
    • 2021
  • Recently, IoT systems are cloud-based, so that continuous and large amounts of data collected from sensor nodes are processed in the data server through the cloud. However, in the centralized configuration of large-scale cloud computing, computational processing must be performed at a physical location where data collection and processing take place, and the need for edge computers to reduce the network load of the cloud system is gradually expanding. In this paper, a cluster system consisting of 6 inexpensive Raspberry Pi boards was constructed to perform fast data processing. And we propose "Kubernetes cluster system(KCS)" for processing large data collection and analysis by model distribution and data pipeline method. To compare the performance of this study, an ensemble model of deep learning was built, and the accuracy, processing performance, and processing time through the proposed KCS system and model distribution were compared and analyzed. As a result, the ensemble model was excellent in accuracy, but the KCS implemented as a data pipeline proved to be superior in processing speed..

Low Cost Cloud-Assisted Peer to Peer Live Streaming

  • Alghazawy, Bahaa Aldeen;Fujita, Satoshi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.4
    • /
    • pp.1732-1750
    • /
    • 2016
  • Recently, Peer-to-Peer (P2P) live streaming assisted by the cloud computing has attracted considerable attention to improve the reliability of the P2P such as the resilience to peer churn and the shortage of upload capacity. The cost of cloud-assistance is comprised of the number of requests issued to the cloud and the amount of data fetched from the cloud. In this paper, we propose three techniques to reduce the cost of such a cloud-assistance.More concretely, in the proposed method, 1) each peer which lost its parent in the overlay can find a new parent by referring to the information registered in the cloud, 2) several peers which proactively fetch chunks from the cloud are dynamically invested, and 3) the number of requests issued to the cloud is reduced by allowing peers to fetch a collection of chunks using a single request. The performance of the proposed method is evaluated by simulation. The simulation results indicate that it reduces the cost of conventional scheme by 46% while guaranteeing the quality of live streaming service.

A Study on Design of Real-time Big Data Collection and Analysis System based on OPC-UA for Smart Manufacturing of Machine Working

  • Kim, Jaepyo;Kim, Youngjoo;Kim, Seungcheon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.13 no.4
    • /
    • pp.121-128
    • /
    • 2021
  • In order to design a real time big data collection and analysis system of manufacturing data in a smart factory, it is important to establish an appropriate wired/wireless communication system and protocol. This paper introduces the latest communication protocol, OPC-UA (Open Platform Communication Unified Architecture) based client/server function, applied user interface technology to configure a network for real-time data collection through IoT Integration. Then, Database is designed in MES (Manufacturing Execution System) based on the analysis table that reflects the user's requirements among the data extracted from the new cutting process automation process, bush inner diameter indentation measurement system and tool monitoring/inspection system. In summary, big data analysis system introduced in this paper performs SPC (statistical Process Control) analysis and visualization analysis with interface of OPC-UA-based wired/wireless communication. Through AI learning modeling with XGBoost (eXtream Gradient Boosting) and LR (Linear Regression) algorithm, quality and visualization analysis is carried out the storage and connection to the cloud.

Building a Private Cloud-Computing System for Greenhouse Control

  • Kim, JoonYong;Lee, Chun Gu;Park, Dong-Hyeok;Park, Heun Dong;Rhee, Joong-Yong
    • Journal of Biosystems Engineering
    • /
    • v.43 no.4
    • /
    • pp.440-444
    • /
    • 2018
  • Purpose: Cloud-computing technology has several advantages, including maintenance, management, accessibility, and computing power. A greenhouse-control system utilizing these advantages was developed using a private cloud-computing system. Methods: A private cloud needs a collection of servers and a suite of software tools to monitor and control cloud-computing resources. In this study, a server farm, operated by OpenStack as a cloud platform, was constructed using servers, and other network devices. Results: The greenhouse-control system was developed according to the fundamental cloud service models: infrastructure as a service, platform as a service, and software as a service. This system has four additional advantages - security, control function, public data use, and data exchange. There are several considerations that must be addressed, such as service level agreement, data ownership, security, and the differences between users. Conclusions: When the advantages are utilized and the considerations are addressed, cloud-computing technology will be beneficial for agricultural use.

An Efficient Cloud Service Quality Performance Management Method Using a Time Series Framework (시계열 프레임워크를 이용한 효율적인 클라우드서비스 품질·성능 관리 방법)

  • Jung, Hyun Chul;Seo, Kwang-Kyu
    • Journal of the Semiconductor & Display Technology
    • /
    • v.20 no.2
    • /
    • pp.121-125
    • /
    • 2021
  • Cloud service has the characteristic that it must be always available and that it must be able to respond immediately to user requests. This study suggests a method for constructing a proactive and autonomous quality and performance management system to meet these characteristics of cloud services. To this end, we identify quantitative measurement factors for cloud service quality and performance management, define a structure for applying a time series framework to cloud service application quality and performance management for proactive management, and then use big data and artificial intelligence for autonomous management. The flow of data processing and the configuration and flow of big data and artificial intelligence platforms were defined to combine intelligent technologies. In addition, the effectiveness was confirmed by applying it to the cloud service quality and performance management system through a case study. Using the methodology presented in this study, it is possible to improve the service management system that has been managed artificially and retrospectively through various convergence. However, since it requires the collection, processing, and processing of various types of data, it also has limitations in that data standardization must be prioritized in each technology and industry.

A study on BSN data collection technique through mobile devices in a cloud environment

  • Hwang, Chigon;Kim, Hyung-Seok;Lee, Jong-Yong;Jung, Kyedong
    • International journal of advanced smart convergence
    • /
    • v.6 no.2
    • /
    • pp.82-88
    • /
    • 2017
  • The data generated by the BSN sensor attached to the human body is mostly mobile. Accordingly, in a mobile cloud environment that processes BSN data, the service should not be fixed in a specific area but be able to support it according to the move. The mobile device must be able to process, filter and transmit the collected BSN data. The cloud server must be able to collect the data processed by the mobile device and provide it as a service. And the transfer of data requires standardized transfer between each device. In this paper, we propose a data delivery method through standard schema when mobile device processes data and provides service in cloud system and a data processing method according to the movement of the mobile device.

A New Approach to Web Data Mining Based on Cloud Computing

  • Zhu, Wenzheng;Lee, Changhoon
    • Journal of Computing Science and Engineering
    • /
    • v.8 no.4
    • /
    • pp.181-186
    • /
    • 2014
  • Web data mining aims at discovering useful knowledge from various Web resources. There is a growing trend among companies, organizations, and individuals alike of gathering information through Web data mining to utilize that information in their best interest. In science, cloud computing is a synonym for distributed computing over a network; cloud computing relies on the sharing of resources to achieve coherence and economies of scale, similar to a utility over a network, and means the ability to run a program or application on many connected computers at the same time. In this paper, we propose a new system framework based on the Hadoop platform to realize the collection of useful information of Web resources. The system framework is based on the Map/Reduce programming model of cloud computing. We propose a new data mining algorithm to be used in this system framework. Finally, we prove the feasibility of this approach by simulation experiment.

An Attack-based Filtering Scheme for Slow Rate Denial-of-Service Attack Detection in Cloud Environment

  • Gutierrez, Janitza Nicole Punto;Lee, Kilhung
    • Journal of Multimedia Information System
    • /
    • v.7 no.2
    • /
    • pp.125-136
    • /
    • 2020
  • Nowadays, cloud computing is becoming more popular among companies. However, the characteristics of cloud computing such as a virtualized environment, constantly changing, possible to modify easily and multi-tenancy with a distributed nature, it is difficult to perform attack detection with traditional tools. This work proposes a solution which aims to collect traffic packets data by using Flume and filter them with Spark Streaming so it is possible to only consider suspicious data related to HTTP Slow Rate Denial-of-Service attacks and reduce the data that will be stored in Hadoop Distributed File System for analysis with the FP-Growth algorithm. With the proposed system, we also aim to address the difficulties in attack detection in cloud environment, facilitating the data collection, reducing detection time and enabling an almost real-time attack detection.

High Rate Denial-of-Service Attack Detection System for Cloud Environment Using Flume and Spark

  • Gutierrez, Janitza Punto;Lee, Kilhung
    • Journal of Information Processing Systems
    • /
    • v.17 no.4
    • /
    • pp.675-689
    • /
    • 2021
  • Nowadays, cloud computing is being adopted for more organizations. However, since cloud computing has a virtualized, volatile, scalable and multi-tenancy distributed nature, it is challenging task to perform attack detection in the cloud following conventional processes. This work proposes a solution which aims to collect web server logs by using Flume and filter them through Spark Streaming in order to only consider suspicious data or data related to denial-of-service attacks and reduce the data that will be stored in Hadoop Distributed File System for posterior analysis with the frequent pattern (FP)-Growth algorithm. With the proposed system, we can address some of the difficulties in security for cloud environment, facilitating the data collection, reducing detection time and consequently enabling an almost real-time attack detection.

The Design of Dynamic Fog Cloud System using mDBaaS

  • Hwang, Chigon;Shin, Hyoyoung;Lee, Jong-Yong;Jung, Kyedong
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.9 no.4
    • /
    • pp.59-66
    • /
    • 2017
  • Cloud computing has evolved into a core computing infrastructure for the internet that encompasses content, as well as communications, applications and commerce. By providing powerful computing and communications capabilities in the palm of the hand everywhere with a variety of smart devices, mobile applications such as virtual reality, sensing and navigation have emerged and radically changed the patterns people live. The data that is generated is getting bigger. Cloud computing, on the other hand, has problems with system load and speed due to the collection, processing and control of remote data. To solve this problem, fog computing has been proposed in which data is collected and processed at an edge. In this paper, we propose a system that dynamically selects a fog server that acts as a cloud in the edge. It serves as a mediator in the cloud, and provides information on the services and systems belonging to the cloud to the mobile device so that the mobile device can act as a fog. When the role of the fog system is complete, we provide it to the cloud to virtualize the fog. The heterogeneous problem of data of mobile nodes can be solved by using mDBaaS (Mobile DataBase as a Service) and we propose a system design method for this.