• Title/Summary/Keyword: Data Stream Process

Search Result 305, Processing Time 0.021 seconds

A High-Speed Data Processing Algorithm for RFID Input Data Stream Using Multi-Buffer (RFID 입력 데이터 스트림에 대한 다중 버퍼 기반의 고속 데이터 처리 알고리즘)

  • Han, Soo;Park, Sang-Hyun;Shin, Seung-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.13 no.2
    • /
    • pp.79-85
    • /
    • 2008
  • The middleware that provides RFID-based ubiquitous application service should process the data inputted constantly in real time, and acquire and deliver the answers of the questions in the application service. Studies for developing a Data Stream Management System(DSMS) has been performed in order to process a large amount of data stream inputted constantly in this way. Previous algorithms on data stream were mostly focused on reducing the average error between the answers of the successive questions and abandon the data according to the priority of them when a load occurs. This article is composed of presenting the necessity of the studies on the DSMS and speedy data processing, suggesting an algorithm to make Possible the speedy data processing using buffers and prompt questions and answers, and testing the performance of the data processing rate and whether a buffer is generated correspondingly to the algorithm suggested, in either a single or a multiple buffer, through simulations.

  • PDF

Data Source Management using weight table in u-GIS DSMS

  • Kim, Sang-Ki;Baek, Sung-Ha;Lee, Dong-Wook;Chung, Warn-Il;Kim, Gyoung-Bae;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.27-33
    • /
    • 2009
  • The emergences of GeoSensor and researches about GIS have promoted many researches of u-GIS. The disaster application coupled in the u-GIS can apply to monitor accident area and to prevent spread of accident. The application needs the u-GIS DSMS technique to acquire, to process GeoSensor data and to integrate them with GIS data. The u-GIS DSMS must process big and large-volume data stream such as spatial data and multimedia data. Due to the feature of the data stream, in u-GIS DSMS, query processing can be delayed. Moreover, as increasing the input rate of data in the area generating events, the network traffic is increased. To solve this problem, in this paper we describe TRIGGER ACTION clause in CQ on the u-GIS DSMS environment and proposes data source management. Data source weight table controls GES information and incoming data rate. It controls incoming data rate as increasing weight at GES of disaster area. Consequently, it can contribute query processing rate and accuracy

  • PDF

Efficient Processing of Multidimensional Sensor stream Data in Digital Marine Vessel (디지털 선박 내 다차원 센서 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Park, Kyung-Woo;Lee, Jin-Seok;Lee, Keong-Hyo;Jung, Min-A;Lee, Sung-Ro
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.5B
    • /
    • pp.794-800
    • /
    • 2010
  • It is necessary to accurate and efficient management for measured digital data from various sensors in digital marine vessel. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. In this paper, We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose that we arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using SVM algorithm. We automatically delete that it isn't necessary to the data from the database and we used to ship diagnosis system for available data. As a result, we obtained to efficient result about 18.3% reduction rate of database using 35,912 data sets.

A Novel Way of Context-Oriented Data Stream Segmentation using Exon-Intron Theory (Exon-Intron이론을 활용한 상황중심 데이터 스트림 분할 방안)

  • Lee, Seung-Hun;Suh, Dong-Hyok
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.16 no.5
    • /
    • pp.799-806
    • /
    • 2021
  • In the IoT environment, event data from sensors is continuously reported over time. Event data obtained in this trend is accumulated indefinitely, so a method for efficient analysis and management of data is required. In this study, a data stream segmentation method was proposed to support the effective selection and utilization of event data from sensors that are continuously reported and received. An identifier for identifying the point at which to start the analysis process was selected. By introducing the role of these identifiers, it is possible to clarify what is being analyzed and to reduce data throughput. The identifier for stream segmentation proposed in this study is a semantic-oriented data stream segmentation method based on the event occurrence of each stream. The existence of identifiers in stream processing can be said to be useful in terms of providing efficiency and reducing its costs in a large-volume continuous data inflow environment.

Statistical process control of dye solution stream using spectrophotometer

  • Lee, Won-Jae;Cho, Gyo-Young
    • Journal of the Korean Data and Information Science Society
    • /
    • v.21 no.6
    • /
    • pp.1289-1303
    • /
    • 2010
  • The need for statistical process control to check the performance of a process is becoming more important in chemical and pharmaceutical industries. This study illustrates the method to determine whether a process is in control and how to produce and interpret control charts. In the experiment, a stream of green dyed water and a stream of pure water were continuously mixed in the process. The concentration of the dye solution was measured before and after the mixer via a spectrophotometer. The in-line mixer provided benefits to the dye and water mixture but not for the stock dye solution. The control charts were analyzed, and the pre-mixer process was in control for both the stock and mixed solutions. The R and X-bar charts showed virtually all of the points within control limits, and there were no patterns in the X-bar charts to suggest nonrandom data. However, the post-mixer process was shown to be out of control. While the R charts showed variability within the control limits, the X-bar charts were out of control and showed a steady increase in values, suggesting that the data was nonrandom. This steady increase in dye concentration was due to discontinuous, non-steady state flow. To improve the experiment in the future, a mixer could be inserted into the stock dye tank. The mixer would ensure that the dye concentration of the stock solution is more uniform prior to entering the pre-mixer ow cell. Overall, this would create a better standard to judge the water and dye mixture data against as well.

Jave based Embedded System Design and Implementation for Real-time Stream Data Processing (Java 기반 실시간 센서 데이터스트림처리 및 임베디드 시스템 구현)

  • Kim, Hyu-Chan;Ko, Wan-Ki;Park, Sang-Yeol
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.4 no.2
    • /
    • pp.1-12
    • /
    • 2008
  • Home network is a technology that provides possibilities of monitoring/controling/mutilating-recognition between optional home network machines in residences. Currently, home network or other networks like entertainment, residential electronic networks are jumbled together with heterogeneous networks in a rampaging condition. In a reality of high expectation for home networks system like the mutual application for various machines, we are required to have the unification technology for conveniences to satisfy expectations. This thesis reflects how to develop Java applications or mutual products based on convenient interfaces actually that process various sensors which create real time data stream in Java platform through Java based sensor data-stream processing embedded middleware design and realization in real time.

Load balancing method of overload prediction for guaranteeing the data completeness in data stream (데이터 스트림 환경에서 데이터 완전도 보장을 위한 과부하 예측 부하 분산 기법)

  • Kim, Young-Ki;Shin, Soong-Sun;Baek, Sung-Ha;Lee, Dong-Wook;Kim, Gyoung-Bae;Bae, Hae-Young
    • Journal of Korea Multimedia Society
    • /
    • v.12 no.9
    • /
    • pp.1242-1251
    • /
    • 2009
  • A DSMS(Data Stream Management System) in ubiquitous environment processes huge data that input from a number of sensor. The existed system is used with a load shedding method that is eliminated with a part of huge data stream when it doesn't process the huge data stream. The Load shedding method has to filter a part of input data. This is because, data completeness or reliability is decreased. In this paper, we proposed the overload prediction load balancing to maintain data completeness when the system has an overload. The proposed method predicts the overload time. and than it is decreased with data loss when achieves the prediction overload time. The performance evaluation shows that the proposed method performs better than the existed method.

  • PDF

Video Quality for DTV Essential Hidden Area Utilization

  • Han, Chan-Ho
    • Journal of Multimedia Information System
    • /
    • v.4 no.1
    • /
    • pp.19-26
    • /
    • 2017
  • The compression of video for both full HD and UHD requires the inclusion of extra vertical lines to every video frame, named as the DTV essential hidden area (DEHA), for the effective functioning of the MPEG-2/4/H encoder, stream, and decoder. However, while the encoding/decoding process is dependent on the DEHA, the DEHA is conventionally viewed as a redundancy in terms of channel utilization or storage efficiency. This paper proposes a block mode DEHA method to more effectively utilize the DEHA. Partitioning video block images and then evenly filling the representative DEHA macroblocks with the average DC coefficient of the active video macroblock can minimize the amount of DEHA data entering the compressed video stream. Theoretically, this process results in smaller DEHA data entering the video stream. Experimental testing of the proposed block mode DEHA method revealed a slight improvement in the quality of the active video. Outside of this technological improvement to video quality, the attractiveness of the proposed DEHA method is also heightened by the ease that it can be implemented with existing video encoders.

Implementation of Real-time Data Stream Processing for Predictive Maintenance of Offshore Plants (해양플랜트의 예지보전을 위한 실시간 데이터 스트림 처리 구현)

  • Kim, Sung-Soo;Won, Jongho
    • Journal of KIISE
    • /
    • v.42 no.7
    • /
    • pp.840-845
    • /
    • 2015
  • In recent years, Big Data has been a topic of great interest for the production and operation work of offshore plants as well as for enterprise resource planning. The ability to predict future equipment performance based on historical results can be useful to shuttling assets to more productive areas. Specifically, a centrifugal compressor is one of the major piece of equipment in offshore plants. This machinery is very dangerous because it can explode due to failure, so it is necessary to monitor its performance in real time. In this paper, we present stream data processing architecture that can be used to compute the performance of the centrifugal compressor. Our system consists of two major components: a virtual tag stream generator and a real-time data stream manager. In order to provide scalability for our system, we exploit a parallel programming approach to use multi-core CPUs to process the massive amount of stream data. In addition, we provide experimental evidence that demonstrates improvements in the stream data processing for the centrifugal compressor.

A Human Movement Stream Processing System for Estimating Worker Locations in Shipyards

  • Duong, Dat Van Anh;Yoon, Seokhoon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.13 no.4
    • /
    • pp.135-142
    • /
    • 2021
  • Estimating the locations of workers in a shipyard is beneficial for a variety of applications such as selecting potential forwarders for transferring data in IoT services and quickly rescuing workers in the event of industrial disasters or accidents. In this work, we propose a human movement stream processing system for estimating worker locations in shipyards based on Apache Spark and TensorFlow serving. First, we use Apache Spark to process location data streams. Then, we design a worker location prediction model to estimate the locations of workers. TensorFlow serving manages and executes the worker location prediction model. When there are requirements from clients, Apache Spark extracts input data from the processed data for the prediction model and then sends it to TensorFlow serving for estimating workers' locations. The worker movement data is needed to evaluate the proposed system but there are no available worker movement traces in shipyards. Therefore, we also develop a mobility model for generating the workers' movements in shipyards. Based on synthetic data, the proposed system is evaluated. It obtains a high performance and could be used for a variety of tasksin shipyards.