• Title/Summary/Keyword: Query Stream

Search Result 153, Processing Time 0.036 seconds

Continuous Query Processing in Data Streams Using Duality of Data and Queries (데이타와 질의의 이원성을 이용한 데이타스트림에서의 연속질의 처리)

  • Lim Hyo-Sang;Lee Jae-Gil;Lee Min-Jae;Whang Kyu-Young
    • Journal of KIISE:Databases
    • /
    • v.33 no.3
    • /
    • pp.310-326
    • /
    • 2006
  • In this paper, we deal with a method of efficiently processing continuous queries in a data stream environment. We classify previous query processing methods into two dual categories - data-initiative and query-initiative - depending on whether query processing is initiated by selecting a data element or a query. This classification stems from the fact that data and queries have been treated asymmetrically. For processing continuous queries, only data-initiative methods have traditionally been employed, and thus, the performance gain that could be obtained by query-initiative methods has been overlooked. To solve this problem, we focus on an observation that data and queries can be treated symmetrically. In this paper, we propose the duality model of data and queries and, based on this model, present a new viewpoint of transforming the continuous query processing problem to a multi-dimensional spatial join problem. We also present a continuous query processing algorithm based on spatial join, named Spatial Join CQ. Spatial Join CQ processes continuous queries by finding the pairs of overlapping regions from a set of data elements and a set of queries defined as regions in the multi-dimensional space. The algorithm achieves the effects of both of the two dual methods by using the spatial join, which is a symmetric operation. Experimental results show that the proposed algorithm outperforms earlier methods by up to 36 times for simple selection continuous queries and by up to 7 times for sliding window join continuous queries.

Efficient k-Nearest Neighbor Query Processing Method for a Large Location Data (대용량 위치 데이터에서 효율적인 k-최근접 질의 처리 기법)

  • Choi, Dojin;Lim, Jongtae;Yoo, Seunghun;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.17 no.8
    • /
    • pp.619-630
    • /
    • 2017
  • With the growing popularity of smart devices, various location based services have been providing to users. Recently, some location based social applications that combine social services and location based services have been emerged. The demands of a k-nearest neighbors(k-NN) query which finds k closest locations from a user location are increased in the location based social network services. In this paper, we propose an approximate k-NN query processing method for fast response time in a large number of users environments. The proposed method performs efficient stream processing using big data distributed processing technologies. In this paper, we also propose a modified grid index method for indexing a large amount of location data. The proposed query processing method first retrieves the related cells by considering a user movement. By doing so, it can make an approximate k results set. In order to show the superiority of the proposed method, we conduct various performance evaluations with the existing method.

An Efficient RFID Business Event Detection Method Using Preprocessing Filtering Scheme (전처리 필터링을 적용한 효율적인 RFID 비즈니스 이벤트 검출 기법)

  • Rho, Jin-Seok;Bok, Kyoung-Soo;Yoo, Jae-Soo
    • Journal of KIISE:Databases
    • /
    • v.35 no.2
    • /
    • pp.143-154
    • /
    • 2008
  • RFID events are large volume of stream data which come out continuously. Many studies have been done to detect a business event in RFID stream. However, the existing methods have many problems which increase unnecessary operations when business events do not satisfy minimum conditions. In this paper, to remove unnecessary operations, we define the minimum condition of business events and propose an efficient method that detects business events only when the minimum condition is satisfied. To check the minimum condition of business events, we register business queries in a query index. We detect business events using the query index and bitmap. It is shown through various experiment that the proposed method outperforms the existing methods.

Research Directions for Efficient Query Processing over Sensor Data Streams (센서 데이터 스트림 환경에서 효율적인 질의처리 연구방향)

  • An, Dong-Chan
    • KSCI Review
    • /
    • v.14 no.2
    • /
    • pp.199-204
    • /
    • 2006
  • The sensor network is a wireless network of the sensor nodes which sensing, computation and communication ability. Each sensor nodes create the data items by sensor nodes above one. Like this feature, the sensor network is similar to distributed data base system. The sensor node of the sensor network is restricted from the power and the memory resources is the biggest weak point and is becoming the important research object. In this paper, We try to see efficient sensor data stream management method and efficient query processing method under the restricted sensor network environment.

  • PDF

H*-tree/H*-cubing-cubing: Improved Data Cube Structure and Cubing Method for OLAP on Data Stream (H*-tree/H*-cubing: 데이터 스트림의 OLAP를 위한 향상된 데이터 큐브 구조 및 큐빙 기법)

  • Chen, Xiangrui;Li, Yan;Lee, Dong-Wook;Kim, Gyoung-Bae;Bae, Hae-Young
    • The KIPS Transactions:PartD
    • /
    • v.16D no.4
    • /
    • pp.475-486
    • /
    • 2009
  • Data cube plays an important role in multi-dimensional, multi-level data analysis. Meeting on-line analysis requirements of data stream, several cube structures have been proposed for OLAP on data stream, such as stream cube, flowcube, S-cube. Since it is costly to construct data cube and execute ad-hoc OLAP queries, more research works should be done considering efficient data structure, query method and algorithms. Stream cube uses H-cubing to compute selected cuboids and store the computed cells in an H-tree, which form the cuboids along popular-path. However, the H-tree layoutis disorderly and H-cubing method relies too much on popular path.In this paper, first, we propose $H^*$-tree, an improved data structure, which makes the retrieval operation in tree structure more efficient. Second, we propose an improved cubing method, $H^*$-cubing, with respect to computing the cuboids that cannot be retrieved along popular-path when an ad-hoc OLAP query is executed. $H^*$-tree construction and $H^*$-cubing algorithms are given. Performance study turns out that during the construction step, $H^*$-tree outperforms H-tree with a more desirable trade-off between time and memory usage, and $H^*$-cubing is better adapted to ad-hoc OLAP querieswith respect to the factors such as time and memory space.

A Query Processing Technique for XML Fragment Stream using XML Labeling (XML 레이블링을 이용한 XML 조각 스트림에 대한 질의 처리 기법)

  • Lee, Sang-Wook;Kim, Jin;Kang, Hyun-Chul
    • Journal of KIISE:Databases
    • /
    • v.35 no.1
    • /
    • pp.67-83
    • /
    • 2008
  • In order to realize ubiquitous computing, it is essential to efficiently use the resources and the computing power of mobile devices. Among others, memory efficiency, energy efficiency, and processing efficiency are required in executing the softwares embedded in mobile devices. In this paper, query processing over XML data in a mobile device where resources are limited is addressed. In a device with limited amount of memory, the techniques of XML. stream query processing need to be employed to process queries over a large volume of XML data Recently, a technique Galled XFrag was proposed whereby XML data is fragmented with the hole-filler model and streamed in fragments for processing. With XFrag, query processing is possible in the mobile device with limited memory without reconstructing the XML data out of its fragment stream. With the hole-filler model, however, memory efficiency is not high because the additional information on holes and fillers needs to be stored. In this paper, we propose a new technique called XFLab whereby XML data is fragmented with the XML labeling scheme which is for representing the structural relationship in XML data, and streamed in fragments for processing. Through implementation and experiments, XML showed that our XFLab outperformed XFrag both in memory usage and processing time.

Frequent Patten Tree based XML Stream Mining (빈발 패턴 트리 기반 XML 스트림 마이닝)

  • Hwang, Jeong-Hee
    • The KIPS Transactions:PartD
    • /
    • v.16D no.5
    • /
    • pp.673-682
    • /
    • 2009
  • XML data are widely used for data representation and exchange on the Web and the data type is an continuous stream in ubiquitous environment. Therefore there are some mining researches related to the extracting of frequent structures and the efficient query processing of XML stream data. In this paper, we propose a mining method to extract frequent structures of XML stream data in recent window based on the sliding window. XML stream data are modeled as a tree set, called XFP_tree and we quickly extract the frequent structures over recent XML data in the XFP_tree.

Data Source Management using weight table in u-GIS DSMS

  • Kim, Sang-Ki;Baek, Sung-Ha;Lee, Dong-Wook;Chung, Warn-Il;Kim, Gyoung-Bae;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.27-33
    • /
    • 2009
  • The emergences of GeoSensor and researches about GIS have promoted many researches of u-GIS. The disaster application coupled in the u-GIS can apply to monitor accident area and to prevent spread of accident. The application needs the u-GIS DSMS technique to acquire, to process GeoSensor data and to integrate them with GIS data. The u-GIS DSMS must process big and large-volume data stream such as spatial data and multimedia data. Due to the feature of the data stream, in u-GIS DSMS, query processing can be delayed. Moreover, as increasing the input rate of data in the area generating events, the network traffic is increased. To solve this problem, in this paper we describe TRIGGER ACTION clause in CQ on the u-GIS DSMS environment and proposes data source management. Data source weight table controls GES information and incoming data rate. It controls incoming data rate as increasing weight at GES of disaster area. Consequently, it can contribute query processing rate and accuracy

  • PDF

Dynamic Load Management Method for Spatial Data Stream Processing on MapReduce Online Frameworks (맵리듀스 온라인 프레임워크에서 공간 데이터 스트림 처리를 위한 동적 부하 관리 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.8
    • /
    • pp.535-544
    • /
    • 2018
  • As the spread of mobile devices equipped with various sensors and high-quality wireless network communications functionsexpands, the amount of spatio-temporal data generated from mobile devices in various service fields is rapidly increasing. In conventional research into processing a large amount of real-time spatio-temporal streams, it is very difficult to apply a Hadoop-based spatial big data system, designed to be a batch processing platform, to a real-time service for spatio-temporal data streams. This paper extends the MapReduce online framework to support real-time query processing for continuous-input, spatio-temporal data streams, and proposes a load management method to distribute overloads for efficient query processing. The proposed scheme shows a dynamic load balancing method for the nodes based on the inflow rate and the load factor of the input data based on the space partition. Experiments show that it is possible to support efficient query processing by distributing the spatial data stream in the corresponding area to the shared resources when load management in a specific area is required.

In-Memory Based Incremental Processing Method for Stream Query Processing in Big Data Environments (빅데이터 환경에서 스트림 질의 처리를 위한 인메모리 기반 점진적 처리 기법)

  • Bok, Kyoungsoo;Yook, Misun;Noh, Yeonwoo;Han, Jieun;Kim, Yeonwoo;Lim, Jongtae;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.16 no.2
    • /
    • pp.163-173
    • /
    • 2016
  • Recently, massive amounts of stream data have been studied for distributed processing. In this paper, we propose an incremental stream data processing method based on in-memory in big data environments. The proposed method stores input data in a temporary queue and compare them with data in a master node. If the data is in the master node, the proposed method reuses the previous processing results located in the node chosen by the master node. If there are no previous results of data in the node, the proposed method processes the data and stores the result in a separate node. We also propose a job scheduling technique considering the load and performance of a node. In order to show the superiority of the proposed method, we compare it with the existing method in terms of query processing time. Our experimental results show that our method outperforms the existing method in terms of query processing time.