• Title/Summary/Keyword: Stream Query Processing

Search Result 124, Processing Time 0.023 seconds

A Transformation Scheme for Continuous Queries on RFID Streaming Data (RFID 스트리밍 데이터 처리를 위한 연속 질의의 변환 기법)

  • Park, Jae-Kwan;Hong, Bong-Hee;Ban, Chae-Hoon
    • The KIPS Transactions:PartD
    • /
    • v.14D no.3 s.113
    • /
    • pp.273-284
    • /
    • 2007
  • RFID middleware systems collect and filter the RFID streaming data gathered continuously by numerous readers in order to process requests from applications. These requests are called continuous queries because they are kept on executing during certain periods. To enhance the performance of the middleware, it is required to build an index to process the continuous queries efficiently. Several approaches of building an index on not data records but queries, called Query Index, are proposed and widely used for evaluating continuous queries over streaming data. The EPCglobal proposed an Event Cycle Specification (ECSpec) model, which is a standard query interface for RFID applications. Continuous queries based on ECSpec consist of a large number of segments for representing the query conditions. The problem with using any of the existing query indexes on these continuous queries is that it takes a long time to build the index because it is necessary to insert a large number of segments into the index. To solve this problem, we propose an Aggregate Transformation that converts a group of segments into a compressed data which is representative of the segments. We compare the performance of a transformed index with the existing query indexes.

Design of Query Processing based on Profiles for Efficient Searching Events (효율적인 이벤트 검색을 위한 프로파일 기반 질의 처리 방법)

  • Kim, ChangHoon;Kim, TaeYoung;Kim, JongMin;Ban, ChaeHoon;Kim, DongHyun
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2009.10a
    • /
    • pp.249-252
    • /
    • 2009
  • Recently, it is possible for users to acquire necessary data easily as the various schemes of the searching information are developed. Since these data rise continuously like stream data, it is required to extract the appropriate data for the user's needs from the mass data on the internet. In the traditional scheme, they are acquired by processing the user queries after the occurred data are stored at a database. However, it is inefficient to process the user queries over the large volume of continuous data by using the traditional scheme. In this paper, we propose the query processing scheme to extract the data efficiently for the user requirements from the large volume of continuous data. On the proposed scheme, we present the Event-Profile Model to define the data occurrence on the internet as the events and the user's requirements as the profiles. We also show the filtering scheme to process the events and the profiles efficiently.

  • PDF

A holistic distributed clustering algorithm based on sensor network (센서 네트워크 기반의 홀리스틱 분산 클러스터링 알고리즘)

  • Chen Ping;Kee-Wook Rim;Nam Ji-Yeun;Lee KyungOh
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2008.11a
    • /
    • pp.874-877
    • /
    • 2008
  • Nowadays the existing data processing systems can only support some simple query for sensor network. It is increasingly important to process the vast data streams in sensor network, and achieve effective acknowledges for users. In this paper, we propose a holistic distributed k-means algorithm for sensor network. In order to verify the effectiveness of this method, we compare it with central k-means algorithm to process the data streams in sensor network. From the evaluation experiments, we can verify that the proposed algorithm is highly capable of processing vast data stream with less computation time. This algorithm prefers to cluster the data streams at the distributed nodes, and therefore it largely reduces redundant data communications compared to the central processing algorithm.

A Study of Similarity Measures on Multidimensional Data Sequences Using Semantic Information (의미 정보를 이용한 다차원 데이터 시퀀스의 유사성 척도 연구)

  • Lee, Seok-Lyong;Lee, Ju-Hong;Chun, Seok-Ju
    • The KIPS Transactions:PartD
    • /
    • v.10D no.2
    • /
    • pp.283-292
    • /
    • 2003
  • One-dimensional time-series data have been studied in various database applications such as data mining and data warehousing. However, in the current complex business environment, multidimensional data sequences (MDS') become increasingly important in addition to one-dimensional time-series data. For example, a video stream can be modeled as an MDS in the multidimensional space with respect to color and texture attributes. In this paper, we propose the effective similarity measures on which the similar pattern retrieval is based. An MDS is partitioned into segments, each of which is represented by various geometric and semantic features. The similarity measures are defined on the basis of these segments. Using the measures, irrelevant segments are pruned from a database with respect to a given query. Both data sequences and query sequences are partitioned into segments, and the query processing is based upon the comparison of the features between data and query segments, instead of scanning all data elements of entire sequences.

Hazelcast Vs. Ignite: Opportunities for Java Programmers

  • Maxim, Bartkov;Tetiana, Katkova;S., Kruglyk Vladyslav;G., Murtaziev Ernest;V., Kotova Olha
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.2
    • /
    • pp.406-412
    • /
    • 2022
  • Storing large amounts of data has always been a big problem from the beginning of computing history. Big Data has made huge advancements in improving business processes by finding the customers' needs using prediction models based on web and social media search. The main purpose of big data stream processing frameworks is to allow programmers to directly query the continuous stream without dealing with the lower-level mechanisms. In other words, programmers write the code to process streams using these runtime libraries (also called Stream Processing Engines). This is achieved by taking large volumes of data and analyzing them using Big Data frameworks. Streaming platforms are an emerging technology that deals with continuous streams of data. There are several streaming platforms of Big Data freely available on the Internet. However, selecting the most appropriate one is not easy for programmers. In this paper, we present a detailed description of two of the state-of-the-art and most popular streaming frameworks: Apache Ignite and Hazelcast. In addition, the performance of these frameworks is compared using selected attributes. Different types of databases are used in common to store the data. To process the data in real-time continuously, data streaming technologies are developed. With the development of today's large-scale distributed applications handling tons of data, these databases are not viable. Consequently, Big Data is introduced to store, process, and analyze data at a fast speed and also to deal with big users and data growth day by day.

Design and Implementation of XML Indexing and Query Scheme Based on Database Concept Structure (데이터베이스의 개념구조에 기반한 XML 문서의 색인 및 질의 스키마의 설계 및 구현)

  • Choo Kyo-Nam;Woo Yo-Seob
    • The KIPS Transactions:PartD
    • /
    • v.13D no.3 s.106
    • /
    • pp.317-324
    • /
    • 2006
  • In this paper, we propose a new indexing technique to solve various queries which have a strong good point not only database indexing schema take advantage of converting from semi-structured data to structured data but also performance is more faster than before. We represent structure information of XML document between nodes of tree that additional numbering information which can be bit-stream without modified structure of XML tree. And, We add in indexing schema searching incidental structure information in the process. In Querying schema, we recover ancestor nodes through give information of node using indexing schema in complete path query expression as well as relative path query expression. Therefore, it takes advantage of making derivative query expression with given query. In this process, we recognize that indexing and querying schema can get searched result set faster and more accurate. Because response time is become shorter by bit operating, when query occur and it just needs information of record set earch node in database.

Historical Sensor Data Management Using Temporal Information (센서 데이터의 시간 정보를 이용한 이력 정보 관리)

  • Lee, Yang-Koo;Ryu, Keun-Ho
    • Journal of Korea Spatial Information System Society
    • /
    • v.10 no.4
    • /
    • pp.97-102
    • /
    • 2008
  • A wireless sensor network consists of many sensors that collect and transmit physical or environmental conditions at different locations to a server continuously. Many researches mainly focus on processing continuous queries on real-time data stream. However, they do not concern the problem of storing the historical data, which is mandatory to the historical queries. In this paper, we propose two time-based storage methods to store the sensor data stream and reduce the managed tuples without any loss of information, which lead to the improvement of the accuracy of query results.

  • PDF

Filtering Method for Analyzing Renewable Energy Stream Data (신재생 에너지 스트림 데이터 분석을 위한 필터링 기법)

  • Jin, Cheng Hao;Li, Xun;Kim, Kyu Ik;Hwang, Mi Yeong;Kim, Sang Yeob;Kim, Kwang Deuk;Ryu, Keun Ho
    • Journal of Convergence Society for SMB
    • /
    • v.1 no.1
    • /
    • pp.39-44
    • /
    • 2011
  • Recently, due to people's incontinent use all over the world, fossil fuels such as coal, oil, and natural gas were nearly to be exhausted and also causes serious environment pollutions. Therefore, there is a strong need to develop solar, wind, hydro, biomass, geothermal to replace fossil fuels to prevent suffering from above problems. Wish advances in sensor technology, such data is collected as a kind of stream data which arrives in an online manner so that it is characterized as high- speed, real-time and unbounded and it requires fast data processing to get the up-to-date results. Therefore, the traditional data processing techniques are not fit to deal with stream data. In this paper, we propose a kalman filter-based algorithm to process renewable stream data.

  • PDF

Query Processing System for Incomplete Sensor Stream Data of in Real-time Sensor Network (실시간 센서 네트워크에서 불완전 센서 스트림 데이터를 위한 질의 처리 시스템)

  • Jang, You-Ho;Lee, Sang-Ho;Kim, Yong-Seung;Oh, Ryum-Duck
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2014.01a
    • /
    • pp.123-124
    • /
    • 2014
  • 무선 센서 네트워크는 센서들을 근거리 네트워크로 연결하여 사용자와 현장의 정보를 실시간으로 연결해 주는 매개체 역할을 한다. 이러한 무선 센서 네트워크는 기존의 컴퓨팅 시스템과는 달리 제한된 자원과 환경 속에서 동작을 해야 하고, 접근이 힘든 곳이나 지속적인 관리가 필요한 지역에서 효율적으로 사용된다. 본 논문에서는 무선 센서네트워크의 제한된 자원 속에서 불완전 스트림 데이터를 효율적으로 정제하고 처리하여 빠르고 정확한 질의어 처리가 가능한 질의 시스템을 제안하였다.

  • PDF