• Title/Summary/Keyword: Stream Query Processing

Search Result 124, Processing Time 0.026 seconds

A Review of Window Query Processing for Data Streams

  • Kim, Hyeon Gyu;Kim, Myoung Ho
    • Journal of Computing Science and Engineering
    • /
    • v.7 no.4
    • /
    • pp.220-230
    • /
    • 2013
  • In recent years, progress in hardware technology has resulted in the possibility of monitoring many events in real time. The volume of incoming data may be so large, that monitoring all individual data might be intractable. Revisiting any particular record can also be impossible in this environment. Therefore, many database schemes, such as aggregation, join, frequent pattern mining, and indexing, become more challenging in this context. This paper surveys the previous efforts to resolve these issues in processing data streams. The emphasis is on specifying and processing sliding window queries, which are supported in many stream processing engines. We also review the related work on stream query processing, including synopsis structures, plan sharing, operator scheduling, load shedding, and disorder control.

The Framework of Stream Data Processing System for Realtime Health Care Service (실시간 헬스케어 서비스를 위한 스트림 데이터 시스템 프레임워크의 설계)

  • Wu, Zejun;Lee, Yeon;Bae, Hae-Young
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2011.06a
    • /
    • pp.21-22
    • /
    • 2011
  • The growth of using smartphone and tablet pc has enabled variety kinds of realtime applications. In these applications, the data which we called data stream is multidimensional, continuous, rapid, and time-varying. However the traditional Database Management System (DBMS) suffers from processing the real time and complex application, in this paper we proposed the framework for CCR Data Stream Server's design and implementation that compiled with Data Stream Database Management System (DSMS) and DBMS in EMR system. The system enables users not only to query stored CCR information from DBMS, but also to execute continues query for the real-time CCR Data Stream.

  • PDF

Spatio-temporal Query Processing Systems for Ubiquitous Environments (유비쿼터스 환경을 위한 시공간 질의 처리 시스템)

  • Lee, Ki-Young;Lim, Myung-Jae;Kim, Kyu-Ho;Kim, Joung-Joon
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.10 no.3
    • /
    • pp.145-152
    • /
    • 2010
  • With the recent development of the ubiquitous computing technology, there are increasing interest and research in technologies such as sensors and RFID related to information recognition and location positioning in various ubiquitous fields. Especially, RTLS(Real-Time Locating Services) dealing with spatio-temporal data is emerging as a promising technology. For these reasons, the ISO/IEC published the RTLS standard specification for compatibility and interoperability in RTLS. Therefore, in this paper, we designed and implemented Spatio-temporal Query Processing Systems for efficiently managing and searching the incoming Spatio-temporal data stream of moving objects. Spatio-temporal Query Processing Systems's spatio-temporal middleware maintains interoperability among heterogeneous devices and guarantees data integrity in query processing through real time processing of unceasing spatio-temporal data streams and two way synchronization of spatio-temporal DBMSs. Web Server uses the SOAP(Simple Object Access Protocol) message between client and server for interoperability and translates client's SOAP message into CQL(Continuous Query Language) of the spatio-temporal middleware. Finally, this thesis proved the utility of the system by applying the spatio-temporal Query Processing Systems to a real-time Locating Services.

A Multi-Query Optimizing Method for Data Stream Similar Queries on Sliding Window (슬라이딩 윈도에서의 데이터 스팀데이터 유사 질의 처리를 위한 다중질의 최적화 기법)

  • Liangbo Li;Yan Li;Song-Sun Shin;Dong-Wook Lee;Weon-Il Chung;Hae-Young Bae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2008.11a
    • /
    • pp.413-416
    • /
    • 2008
  • In the presence of multiple continuous queries, multi-query optimizing is a new challenge to process multiple stream data in real-time. So, in this paper, we proposed an approach to optimize multi-query of sliding window on network traffic data streams and do some comparisons to traditional queries without optimizing. We also detail some method of scheduling on different data streams, while different scheduling made different results. We test the results on variety of multi-query processing schedule, and proofed the proposed method is effectively optimized the data stream similar multi-queries.

Mining of Frequent Structures over Streaming XML Data (스트리밍 XML 데이터의 빈발 구조 마이닝)

  • Hwang, Jeong-Hee
    • The KIPS Transactions:PartD
    • /
    • v.15D no.1
    • /
    • pp.23-30
    • /
    • 2008
  • The basic research of context aware in ubiquitous environment is an internet technique and XML. The XML data of continuous stream type are popular in network application through the internet. And also there are researches related to query processing for streaming XML data. As a basic research to efficiently query, we propose not only a labeled ordered tree model representing the XML but also a mining method to extract frequent structures from streaming XML data. That is, XML data to continuously be input are modeled by a stream tree which is called by XFP_tree and we exactly extract the frequent structures from the XFP_tree of current window to mine recent data. The proposed method can be applied to the basis of the query processing and index method for XML stream data.

Stream Data Processing based on Sliding Window at u-Health System (u-Health 시스템에서 슬라이딩 윈도우 기반 스트림 데이터 처리)

  • Kim, Tae-Yeun;Song, Byoung-Ho;Bae, Sang-Hyun
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.4 no.2
    • /
    • pp.103-110
    • /
    • 2011
  • It is necessary to accurate and efficient management for measured digital data from sensors in u-health system. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using backpropagation algorithm. As a result, we obtained to efficient result about 18.3% reduction rate of database using 14,324 data sets.

Query Optimization for an Advanced Keyword Search on Relational Data Stream (관계형 데이터 스트림에서 고급 키워드 검색을 위한 질의 최적화)

  • Joo, Jin-Ung;Kim, Hak-Soo;Hwang, Jin-Ho;Son, Jin-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.16D no.6
    • /
    • pp.859-870
    • /
    • 2009
  • Despite the surge in the research for keyword search method over relational database, only little attention has been devoted to studying on relational data stream.The research for keyword search over relational data stream is intense interest because streaming data is recently a major research topic of growing interest in the data management. In this regard we first analyze the researches related to keyword search methodover relational data stream, and then this paper focuses on the method of minimizing the join cost occurred while processing keyword search queries. As a result, we propose an advanced keyword search method that can yield more meaningful results for users on relational data streams. We also propose a query optimization method using layered-clustering for efficient query processing.

Memory Efficient Query Processing over Dynamic XML Fragment Stream (동적 XML 조각 스트림에 대한 메모리 효율적 질의 처리)

  • Lee, Sang-Wook;Kim, Jin;Kang, Hyun-Chul
    • The KIPS Transactions:PartD
    • /
    • v.15D no.1
    • /
    • pp.1-14
    • /
    • 2008
  • This paper is on query processing in the mobile devices where memory capacity is limited. In case that a query against a large volume of XML data is processed in such a mobile device, techniques of fragmenting the XML data into chunks and of streaming and processing them are required. Such techniques make it possible to process queries without materializing the XML data in its entirety. The previous schemes such as XFrag[4], XFPro[5], XFLab[6] are not scalable with respect to the increase of the size of the XML data because they lack proper memory management capability. After some information on XML fragments necessary for query processing is stored, it is not deleted even after it becomes of no use. As such, when the XML fragments are dynamically generated and infinitely streamed, there could be no guarantee of normal completion of query processing. In this paper, we address scalability of query processing over dynamic XML fragment stream, proposing techniques of deleting information on XML fragments accumulated during query processing in order to extend the previous schemes. The performance experiments through implementation showed that our extended schemes considerably outperformed the previous ones in memory efficiency and scalability with respect to the size of the XML data.

Spatial Operation Allocation Scheme over Common Query Regions for Distributed Spatial Data Stream Processing (분산 공간 데이터 스트림 처리에서 질의 영역의 겹침을 고려한 공간 연산 배치 기법)

  • Chung, Weon-Il
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.13 no.6
    • /
    • pp.2713-2719
    • /
    • 2012
  • According to increasing of various location-based services, distributed data stream processing techniques have been widely studied to provide high scalability and availability. In previous researches, in order to balance the load of distributed nodes, the geographic characteristics of spatial data stream are not considered. For this reason, distributed operations for adjacent spatial regions increases the overall system load. We propose a operation allocation scheme considering the characteristics of spatial operations to effectively processing spatial data stream in distributed computing environments. The proposed method presents the efficient share maximizing approach that preferentially distributes spatial operations sharing the common query regions to the same node in order to separate the adjacent spatial operations on overlapped regions.

Preprocessing Method for Handling Multi-Way Join Continuous Queries over Data Streams (데이터 스트림에서 다중 조인 연속질의의 효과적인 처리를 위한 전처리 기법)

  • Seo, Ki-Yeon;Lee, Joo-Il;Lee, Won-Suk
    • Journal of Internet Computing and Services
    • /
    • v.13 no.3
    • /
    • pp.93-105
    • /
    • 2012
  • A data stream is a series of tuples which are generated in real-time, incessant, immense, and volatile manner. As new information technologies are actively emerging, stream processing methods are being needed to efficiently handle data streams. Especially, finding out an efficient evaluation for a multi-way join would make outstanding contributions toward improving the performance of a data stream management system because a join operation is one of the most resource-consuming operators for evaluating queries. In this paper, in order to evaluate efficiently a multi-way join continuous query, we propose a novel method to decrease the cost of a query by eliminating unsuccessful intermediate results. For this, we propose a matrix-based structure for monitoring data streams and estimate the number of final result tuples of the query and find out unsuccessful tuples by matrix multiplication operations. And then using these information, we process efficiently a multi-way join continuous query by filtering out the unsuccessful tuples in advance before actual evaluation of the query.