• Title/Summary/Keyword: Stream Processing

Search Result 720, Processing Time 0.027 seconds

TIM: A Trapdoor Hash Function-based Authentication Mechanism for Streaming Applications

  • Seo, Seog Chung;Youn, Taek-Young
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.6
    • /
    • pp.2922-2945
    • /
    • 2018
  • Achieving efficient authentication is a crucial issue for stream data commonly seen in content delivery, peer-to-peer, and multicast/broadcast networks. Stream authentication mechanisms need to be operated efficiently at both sender-side and receiver-side at the same time because of the properties of stream data such as real-time and delay-sensitivity. Until now, many stream authentication mechanisms have been proposed, but they are not efficient enough to be used in stream applications where the efficiency for sender and receiver sides are required simultaneously since most of them could achieve one of either sender-side and receiver-side efficiency. In this paper, we propose an efficient stream authentication mechanism, so called TIM, by integrating Trapdoor Hash Function and Merkle Hash Tree. Our construction can support efficient streaming data processing at both sender-side and receiver-side at the same time differently from previously proposed other schemes. Through theoretical and experimental analysis, we show that TIM can provide enhanced performance at both sender and receiver sides compared with existing mechanisms. Furthermore, TIM provides an important feature for streaming authentication, the resilience against transmission loss, since each data block can be verified with authentication information contained in itself.

Frequent Items Mining based on Regression Model in Data Streams (스트림 데이터에서 회귀분석에 기반한 빈발항목 예측)

  • Lee, Uk-Hyun
    • The Journal of the Korea Contents Association
    • /
    • v.9 no.1
    • /
    • pp.147-158
    • /
    • 2009
  • Recently, the data model in stream data environment has massive, continuous, and infinity properties. However the stream data processing like query process or data analysis is conducted using a limited capacity of disk or memory. In these environment, the traditional frequent pattern discovery on transaction database can be performed because it is difficult to manage the information continuously whether a continuous stream data is the frequent item or not. In this paper, we propose the method which we are able to predict the frequent items using the regression model on continuous stream data environment. We can use as a prediction model on indefinite items by constructing the regression model on stream data. We will show that the proposed method is able to be efficiently used on stream data environment through a variety of experiments.

Stream Data Analysis of the Weather on the Location using Principal Component Analysis (주성분 분석을 이용한 지역기반의 날씨의 스트림 데이터 분석)

  • Kim, Sang-Yeob;Kim, Kwang-Deuk;Bae, Kyoung-Ho;Ryu, Keun-Ho
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.28 no.2
    • /
    • pp.233-237
    • /
    • 2010
  • The recent advance of sensor networks and ubiquitous techniques allow collecting and analyzing of the data which overcome the limitation imposed by time and space in real-time for making decisions. Also, analysis and prediction of collected data can support useful and necessary information to users. The collected data in sensor networks environment is the stream data which has continuous, unlimited and sequential properties. Because of the continuous, unlimited and large volume properties of stream data, managing stream data is difficult. And the stream data needs dynamic processing method because of the memory constraint and access limitation. Accordingly, we analyze correlation stream data using principal component analysis. And using result of analysis, it helps users for making decisions.

Causality join query processing for data stream by spatio-temporal sliding window (시공간 슬라이딩윈도우기법을 이용한 데이터스트림의 인과관계 결합질의처리방법)

  • Kwon, O-Je;Li, Ki-Joune
    • Spatial Information Research
    • /
    • v.16 no.2
    • /
    • pp.219-236
    • /
    • 2008
  • Data stream collected from sensors contain a large amount of useful information including causality relationships. The causality join query for data stream is to retrieve a set of pairs (cause, effect) from streams of data. A part of causality pairs may however be lost from the query result, due to the delay from sensors to a data stream management system, and the limited size of sliding windows. In this paper, we first investigate spatial, temporal, and spatio-temporal aspects of the causality join query for data stream. Second, we propose several strategies for sliding window management based on these observations. The accuracy of the proposed strategies is studied by intensive experiments, and the result shows that we improve the accuracy of causality join query in data stream from simple FIFO strategy.

  • PDF

An Efficient Complex Event Processing Algorithm based on INFA-HTS for Out-of-order RFID Event Streams

  • Wang, Jianhua;Wang, Tao;Cheng, Lianglun;Lu, Shilei
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.9
    • /
    • pp.4307-4325
    • /
    • 2016
  • With the aim of solving the problems of long processing times, high memory consumption and low event throughput in the current processing approaches in out-of-order RFID event streams, an efficient complex event processing method based on INFA-HTS (Improved Nondeterministic Finite Automaton-Hash Table Structure) is presented in this paper. The contribution of this paper lies in the fact that we use INFA and HTS to successfully realize the detection of complex events for out-of-order RFID event streams. Specifically, in our scheme, to detect the disorder of out-of-order event streams, we expand the traditional NFA model into a new INFA model to capture the related RFID primitive events from the out-of-order event stream. To high-efficiently manage the large intermediate capturing results, we use the HTS to store and process them. As a result, these problems in the existing methods can be effectively solved by our scheme. The simulation results of our experiments show that our proposed method in this paper outperforms some of the current general processing approaches used to process out-of-order RFID event streams.

Spatio-temporal Query Processing Systems for Ubiquitous Environments

  • Kim, Jeong Joon;Kang, Jeong Jin;Rothwell, Edward J.;Lee, Ki Young
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.5 no.2
    • /
    • pp.1-4
    • /
    • 2013
  • With the recent development of the ubiquitous computing technology, there are increasing interest and research in technologies such as sensors and RFID related to information recognition and location positioning in various ubiquitous fields. Especially, RTLS (Real-Time Locating Services) dealing with spatio-temporal data is emerging as a promising technology. For these reasons, the ISO/IEC published RTLS standard specification for compatibility and interoperability in RTLS. Therefore, in this paper, we designed and implemented Spatio-temporal Query Processing Systems for efficiently managing and searching the incoming Spatio-temporal data stream of moving objects. Spatio-temporal Query Processing Systems's spatio-temporal middleware maintains interoperability among heterogeneous devices and guarantees data integrity in query processing through real time processing of unceasing spatio-temporal data streams and two way synchronization of spatio-temporal DBMSs. Web Server uses the SOAP(Simple Object Access Protocol) message between client and server for interoperability and translates client's SOAP message into CQL(Continuous Query Language) of the spatio-temporal middleware.

An Efficient Complex Event Processing Algorithm based on Multipattern Sharing for Massive Manufacturing Event Streams

  • Wang, Jianhua;Lan, Yubin;Lu, Shilei;Cheng, Lianglun
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.3
    • /
    • pp.1385-1402
    • /
    • 2019
  • Quickly picking up some valuable information from massive manufacturing event stream usually faces with the problem of long detection time, high memory consumption and low detection efficiency due to its stream characteristics of large volume, high velocity, many variety and small value. Aiming to solve the problem above for the current complex event processing methods because of not sharing detection during the detecting process for massive manufacturing event streams, an efficient complex event processing method based on multipattern sharing is presented in this paper. The achievement of this paper lies that a multipattern sharing technology is successfully used to realize the quick detection of complex event for massive manufacturing event streams. Specially, in our scheme, we firstly use pattern sharing technology to merge all the same prefix, suffix, or subpattern that existed in single pattern complex event detection models into a multiple pattern complex event detection model, then we use the new detection model to realize the quick detection for complex events from massive manufacturing event streams, as a result, our scheme can effectively solve the problems above by reducing lots of redundant building, storing, searching and calculating operations with pattern sharing technology. At the end of this paper, we use some simulation experiments to prove that our proposed multiple pattern processing scheme outperforms some general processing methods in current as a whole.

Resource Efficient AI Service Framework Associated with a Real-Time Object Detector

  • Jun-Hyuk Choi;Jeonghun Lee;Kwang-il Hwang
    • Journal of Information Processing Systems
    • /
    • v.19 no.4
    • /
    • pp.439-449
    • /
    • 2023
  • This paper deals with a resource efficient artificial intelligence (AI) service architecture for multi-channel video streams. As an AI service, we consider the object detection model, which is the most representative for video applications. Since most object detection models are basically designed for a single channel video stream, the utilization of the additional resource for multi-channel video stream processing is inevitable. Therefore, we propose a resource efficient AI service framework, which can be associated with various AI service models. Our framework is designed based on the modular architecture, which consists of adaptive frame control (AFC) Manager, multiplexer (MUX), adaptive channel selector (ACS), and YOLO interface units. In order to run only a single YOLO process without regard to the number of channels, we propose a novel approach efficiently dealing with multi-channel input streams. Through the experiment, it is shown that the framework is capable of performing object detection service with minimum resource utilization even in the circumstance of multi-channel streams. In addition, each service can be guaranteed within a deadline.

A Proposal of Event Stream Processing Frameworks applicable to Asynchronous-based Microservice (비동기 기반 마이크로 서비스에 적용 가능한 이벤트 스트림 처리 프레임워크 제안)

  • Park, Sang Il
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.17 no.2
    • /
    • pp.45-50
    • /
    • 2017
  • Micro-service Architecture is a service architecture optimized for large-scale distributed systems such as real-time realistic broadcasting systems, so that are fiercely adopted by Global leading service platform vendors such as Netflix and Twitter due to the merit of horizontal performance scalability enabling the scale-out technique. In addition, micro-service architecture makes it possible to execute image processing and real-time data analysis using an asynchronous-based processing that are difficult to handle in Web API such as REST. In this paper, an event stream processing framework applicable to asynchronous based micro services is proposed in the sense that the accountability of event processing order is not guaranteed in the events such as IoT sensor data analysis or cloud-based image editing because these are the situations where the real-time media editing generates multiple event streams and asynchronous processes in the platform.

A Query Preprocessing Tool for Performance Improvement in Complex Event Stream Query Processing (복합 이벤트 스트림 질의 처리 성능 개선을 위한 질의 전처리 도구)

  • Choi, Joong-Hyun;Cho, Eun-Sun;Lee, Kang-Woo
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.8
    • /
    • pp.513-523
    • /
    • 2015
  • A complex event processing system, becoming useful in real life domains, efficiently processes stream of continuous events like sensor data from IoT systems. However, those systems do not work well on some types of queries yet, so that programmers should be careful about that. For instance, they do not sufficiently provide detailed guide to choose efficient queries among the almost same meaning queries. In this paper, we propose an query preprocessing tool for event stream processing systems, which helps programmers by giving them the hints to improve performance whenever their queries fall in any possible bad formats in the performance sense. We expect that our proposed module would be a big help to increases productivity of writing programs where debugging, testing, and performance tuning are not straightforward.