• Title/Summary/Keyword: Stream Query Processing

Search Result 124, Processing Time 0.024 seconds

A Query Preprocessing Tool for Performance Improvement in Complex Event Stream Query Processing (복합 이벤트 스트림 질의 처리 성능 개선을 위한 질의 전처리 도구)

  • Choi, Joong-Hyun;Cho, Eun-Sun;Lee, Kang-Woo
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.8
    • /
    • pp.513-523
    • /
    • 2015
  • A complex event processing system, becoming useful in real life domains, efficiently processes stream of continuous events like sensor data from IoT systems. However, those systems do not work well on some types of queries yet, so that programmers should be careful about that. For instance, they do not sufficiently provide detailed guide to choose efficient queries among the almost same meaning queries. In this paper, we propose an query preprocessing tool for event stream processing systems, which helps programmers by giving them the hints to improve performance whenever their queries fall in any possible bad formats in the performance sense. We expect that our proposed module would be a big help to increases productivity of writing programs where debugging, testing, and performance tuning are not straightforward.

Research Directions for Efficient Query Processing over Sensor Data Streams (센서 데이터 스트림 환경에서 효율적인 질의처리 연구방향)

  • An, Dong-Chan
    • KSCI Review
    • /
    • v.14 no.2
    • /
    • pp.199-204
    • /
    • 2006
  • The sensor network is a wireless network of the sensor nodes which sensing, computation and communication ability. Each sensor nodes create the data items by sensor nodes above one. Like this feature, the sensor network is similar to distributed data base system. The sensor node of the sensor network is restricted from the power and the memory resources is the biggest weak point and is becoming the important research object. In this paper, We try to see efficient sensor data stream management method and efficient query processing method under the restricted sensor network environment.

  • PDF

Efficient Processing of Multidimensional Sensor stream Data in Digital Marine Vessel (디지털 선박 내 다차원 센서 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Park, Kyung-Woo;Lee, Jin-Seok;Lee, Keong-Hyo;Jung, Min-A;Lee, Sung-Ro
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.5B
    • /
    • pp.794-800
    • /
    • 2010
  • It is necessary to accurate and efficient management for measured digital data from various sensors in digital marine vessel. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. In this paper, We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose that we arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using SVM algorithm. We automatically delete that it isn't necessary to the data from the database and we used to ship diagnosis system for available data. As a result, we obtained to efficient result about 18.3% reduction rate of database using 35,912 data sets.

A Continuous Query Processing System for XML Stream Data (XML 스트림 데이터에 대한 연속 질의 처리 시스템)

  • Han Seungchul;Kang Hyunchul
    • The KIPS Transactions:PartD
    • /
    • v.11D no.7 s.96
    • /
    • pp.1375-1384
    • /
    • 2004
  • Streaming data processing is an area of interest with much research under way. There has been increasing attention on the demands for efficient processing of streaming data produced in the application areas such as monitoring and sensor network. We have developed a continuous query processing system for streaming data and evaluated its performance in this paper. XML, the standard for data exchange on the web, is used as the model for the streaming data and the XQuery appended with a time interval is adopted as the query language for expressing con-tinuous queries. In the proposed system, the result is produced through background processing and materialized for reute in subsequent query processing. Through a detailed set of performance experiments, we shoed the effectiveness of the proposed system.

Adaptive Memory Management Method based on Utilization Ratio to Process Continuous Query (연속질의의 처리를 위한 이용률 기반의 적응적 메모리 관리 기법)

  • Baek, Sung-Ha;Lee, Dong-Wook;Eo, Sang-Hun;Chung, Weon-Il;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.79-88
    • /
    • 2009
  • The volume of memory to store real-time data stream is varied dynamically. Continuous queries processing the data stream must manage the storage volume dynamically. In previous research, according to current volume of data a general memory manager which allocates and releases memory by a page unit is researched.However, the method frequently executes page allocation and release to store data stream. Moreover, particularly delayed queries can monopolize many of pages because the method directly allocates pages when a query has not enough memory. Focusing on the problems in memory management systems, this research proposes a memory management method which reduces the frequency of allocation and release and uniformly distributes pages for queries. The method can reduce the frequency of allocation and release through allocation based on utilization ratio of pages in each query and prevent memory monopoly through memory allocation which considers query delay.

  • PDF

Optimizing Multi-way Join Query Over Data Streams (데이타 스트림에서의 다중 조인 질의 최적화 방법)

  • Park, Hong-Kyu;Lee, Won-Suk
    • Journal of KIISE:Databases
    • /
    • v.35 no.6
    • /
    • pp.459-468
    • /
    • 2008
  • A data stream which is a massive unbounded sequence of data elements continuously generated at a rapid rate. Many recent research activities for emerging applications often need to deal with the data stream. Such applications can be web click monitoring, sensor data processing, network traffic analysis. telephone records and multi-media data. For this. data processing over a data stream are not performed on the stored data but performed the newly updated data with pre-registered queries, and then return a result immediately or periodically. Recently, many studies are focused on dealing with a data stream more than a stored data set. Especially. there are many researches to optimize continuous queries in order to perform them efficiently. This paper proposes a query optimization algorithm to manage continuous query which has multiple join operators(Multi-way join) over data streams. It is called by an Extended Greedy query optimization based on a greedy algorithm. It defines a join cost by a required operation to compute a join and an operation to process a result and then stores all information for computing join cost and join cost in the statistics catalog. To overcome a weak point of greedy algorithm which has poor performance, the algorithm selects the set of operators with a small lay, instead of operator with the smallest cost. The set is influenced the accuracy and execution time of the algorithm and can be controlled adaptively by two user-defined values. Experiment results illustrate the performance of the EGA algorithm in various stream environments.

Load Shedding Method based on Grid Hash to Improve Accuracy of Spatial Sliding Window Aggregate Queries (공간 슬라이딩 윈도우 집계질의의 정확도 향상을 위한 그리드 해쉬 기반의 부하제한 기법)

  • Baek, Sung-Ha;Lee, Dong-Wook;Kim, Gyoung-Bae;Chung, Weon-Il;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.89-98
    • /
    • 2009
  • As data stream is entered into system continuously and the memory space is limited, the data exceeding the memory size cannot be processed. In order to solve the problem, load shedding methods which drop a part of data to prevent exceeding the storage space have been researched. Generally, a traditional load shedding method uses random sampling with optimized rate according to data deviation. The method samples data not to distinguish those used in spatial query because the method uses only a random sampling with optimized rate according to data deviation. Therefore, the accuracy of query was reduced in u-GIS environment including spatial query. In this paper, we researched a new load shedding method improving accuracy of the query in u-GIS environment which runs spatial query and aspatial query simultaneously. The method uses a new sampling method that samples data having low probability used in query. Therefore proposed method improves spatial query accuracy and query processing speed as applying spatial filtering operation to sampling operator.

  • PDF

Frequent Patten Tree based XML Stream Mining (빈발 패턴 트리 기반 XML 스트림 마이닝)

  • Hwang, Jeong-Hee
    • The KIPS Transactions:PartD
    • /
    • v.16D no.5
    • /
    • pp.673-682
    • /
    • 2009
  • XML data are widely used for data representation and exchange on the Web and the data type is an continuous stream in ubiquitous environment. Therefore there are some mining researches related to the extracting of frequent structures and the efficient query processing of XML stream data. In this paper, we propose a mining method to extract frequent structures of XML stream data in recent window based on the sliding window. XML stream data are modeled as a tree set, called XFP_tree and we quickly extract the frequent structures over recent XML data in the XFP_tree.

Dynamic Load Shedding Scheme based on Input Rate of Spatial Data Stream and Data Density (공간 데이터스트림의 입력 빈도와 데이터 밀집도 기반의 동적 부하제한 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.16 no.3
    • /
    • pp.2158-2164
    • /
    • 2015
  • In u-GIS environments, various load shedding techniques have been researched in order to balance loads caused by input spatial data streams. However, typical load shedding methods on aspatial data lack regard for characteristics of spatial data, also previous load shedding approaches on spatial, which still lack regard for spatial data density or dynamic input data stream, give rise to troubles on spatial query processing performance and accuracy. Therefore, dynamic load shedding scheme over spatial data stream is proposed through stored spatial data deviation and load ratio of input data stream in order to improve spatial continuous query accuracy and performance in u-GIS environment. In proposed scheme, input data which are a big probability related to spatial continuous query may be a strong chance to be dropped relatively.

A Data Processing Mechanism in Sensor Network Environment (센서 네트워크 환경에서의 데이터 처리 메커니즘)

  • Park, Dae-Hyun;Kim, Young-Jun;Lee, Jeong-Hoom;Chong, Il-Young
    • Proceedings of the IEEK Conference
    • /
    • 2007.07a
    • /
    • pp.133-134
    • /
    • 2007
  • The effective data processing mechanism in the sensor network means data stream model and real-time query processing model for real-time processing of stream data. This mechanism can improve satisfaction of users and reduce delay rate of data processing. In this paper, we explain the problem which is occurred when users need to search certain information among information of stream data and describe reduction model of delay rate according to data transmission.

  • PDF