• Title/Summary/Keyword: Query Reduction

Search Result 46, Processing Time 0.023 seconds

Cost Analysis of Window Memory Relocation for Data Stream Processing (데이터 스트림 처리를 위한 윈도우 메모리 재배치의 비용 분석)

  • Lee, Sang-Don
    • The Journal of the Korea Contents Association
    • /
    • v.8 no.4
    • /
    • pp.48-54
    • /
    • 2008
  • This paper analyzes cost tradeoffs between memory usage and computation for window-based operators in data stream environments. It identifies generic operator network constructs, and sets up a cost model for the estimation of the expected memory reduction and the computation overheads when window memory relocations are applied to each operator network construct. This cost model helps to identify the utility of window memory relocations. It also helps to apply window memory relocation to improve a query execution plan to save memory usage. The proposed approach contributes to expand the scope of query processing and optimization in data stream environments. It also provides a basis to develop a cost estimation model for the query optimization using window memory relocations.

Stream Data Processing based on Sliding Window at u-Health System (u-Health 시스템에서 슬라이딩 윈도우 기반 스트림 데이터 처리)

  • Kim, Tae-Yeun;Song, Byoung-Ho;Bae, Sang-Hyun
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.4 no.2
    • /
    • pp.103-110
    • /
    • 2011
  • It is necessary to accurate and efficient management for measured digital data from sensors in u-health system. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using backpropagation algorithm. As a result, we obtained to efficient result about 18.3% reduction rate of database using 14,324 data sets.

A PCA-based Data Stream Reduction Scheme for Sensor Networks (센서 네트워크를 위한 PCA 기반의 데이터 스트림 감소 기법)

  • Fedoseev, Alexander;Choi, Young-Hwan;Hwang, Een-Jun
    • Journal of Internet Computing and Services
    • /
    • v.10 no.4
    • /
    • pp.35-44
    • /
    • 2009
  • The emerging notion of data stream has brought many new challenges to the research communities as a consequence of its conceptual difference with conventional concepts of just data. One typical example is data stream processing in sensor networks. The range of data processing considerations in a sensor network is very wide, from physical resource restrictions such as bandwidth, energy, and memory to the peculiarities of query processing including continuous and specific types of queries. In this paper, as one of the physical constraints in data stream processing, we consider the problem of limited memory and propose a new scheme for data stream reduction based on the Principal Component Analysis (PCA) technique. PCA can transform a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables. We adapt PCA for the data stream of a sensor network assuming the cooperation of a query engine (or application) with a network base station. Our method exploits the spatio-temporal correlation among multiple measurements from different sensors. Finally, we present a new framework for data processing and describe a number of experiments under this framework. We compare our scheme with the wavelet transform and observe the effect of time stamps on the compression ratio. We report on some of the results.

  • PDF

A Bottom up Filtering Tuple Selection Method for Continuous Skyline Query Processing in Sensor Networks (센서 네트워크에서 연속 스카이라인 질의 처리를 위한 상향식 필터링 투플 선정 방법)

  • Sun, Jin-Ho;Chung, Chin-Wan
    • Journal of KIISE:Databases
    • /
    • v.36 no.4
    • /
    • pp.280-291
    • /
    • 2009
  • Skyline Query processing is important to wireless sensor applications in order to process multi-dimensional data efficiently. Most skyline researches about sensor network focus on minimizing the energy consumption due to the battery powered constraints. In order to reduce energy consumption, Filtering Method is proposed. Most existing researches have assumed a snapshot skyline query processing and do not consider continuous queries and use data generated in ancestor node. In this paper, we propose an energy efficient method called Bottom up filtering tuple selection for continuous skyline query processing. Past skyline data generated in child nodes are stored in each sensor node and is used when choosing filtering tuple. We also extend the algorithms, called Support filtering tuple(SFT) that is used when we choose the additional filtering tuple. There is a temporal correlation between previous sensing data and recent sensing data. Thus, Based on past data, we estimate current data. By considering this point, we reduce the unnecessary communication cost. The experimental results show that our method outperforms the existing methods in terms of both data reduction rate(DRR) and total communication cost.

Efficient Multi-Step k-NN Search Methods Using Multidimensional Indexes in Large Databases (대용량 데이터베이스에서 다차원 인덱스를 사용한 효율적인 다단계 k-NN 검색)

  • Lee, Sanghun;Kim, Bum-Soo;Choi, Mi-Jung;Moon, Yang-Sae
    • Journal of KIISE
    • /
    • v.42 no.2
    • /
    • pp.242-254
    • /
    • 2015
  • In this paper, we address the problem of improving the performance of multi-step k-NN search using multi-dimensional indexes. Due to information loss by lower-dimensional transformations, existing multi-step k-NN search solutions produce a large tolerance (i.e., a large search range), and thus, incur a large number of candidates, which are retrieved by a range query. Those many candidates lead to overwhelming I/O and CPU overheads in the postprocessing step. To overcome this problem, we propose two efficient solutions that improve the search performance by reducing the tolerance of a range query, and accordingly, reducing the number of candidates. First, we propose a tolerance reduction-based (approximate) solution that forcibly decreases the tolerance, which is determined by a k-NN query on the index, by the average ratio of high- and low-dimensional distances. Second, we propose a coefficient control-based (exact) solution that uses c k instead of k in a k-NN query to obtain a tigher tolerance and performs a range query using this tigher tolerance. Experimental results show that the proposed solutions significantly reduce the number of candidates, and accordingly, improve the search performance in comparison with the existing multi-step k-NN solution.

Efficient Image Retrieval using Minimal Spatial Relationships (최소 공간관계를 이용한 효율적인 이미지 검색)

  • Lee, Soo-Cheol;Hwang, Een-Jun;Byeon, Kwang-Jun
    • Journal of KIISE:Databases
    • /
    • v.32 no.4
    • /
    • pp.383-393
    • /
    • 2005
  • Retrieval of images from image databases by spatial relationship can be effectively performed through visual interface systems. In these systems, the representation of image with 2D strings, which are derived from symbolic projections, provides an efficient and natural way to construct image index and is also an ideal representation for the visual query. With this approach, retrieval is reduced to matching two symbolic strings. However, using 2D-string representations, spatial relationships between the objects in the image might not be exactly specified. Ambiguities arise for the retrieval of images of 3D scenes. In order to remove ambiguous description of object spatial relationships, in this paper, images are referred by considering spatial relationships using the spatial location algebra for the 3D image scene. Also, we remove the repetitive spatial relationships using the several reduction rules. A reduction mechanism using these rules can be used in query processing systems that retrieve images by content. This could give better precision and flexibility in image retrieval.

The Kernel Trick for Content-Based Media Retrieval in Online Social Networks

  • Cha, Guang-Ho
    • Journal of Information Processing Systems
    • /
    • v.17 no.5
    • /
    • pp.1020-1033
    • /
    • 2021
  • Nowadays, online or mobile social network services (SNS) are very popular and widely spread in our society and daily lives to instantly share, disseminate, and search information. In particular, SNS such as YouTube, Flickr, Facebook, and Amazon allow users to upload billions of images or videos and also provide a number of multimedia information to users. Information retrieval in multimedia-rich SNS is very useful but challenging task. Content-based media retrieval (CBMR) is the process of obtaining the relevant image or video objects for a given query from a collection of information sources. However, CBMR suffers from the dimensionality curse due to inherent high dimensionality features of media data. This paper investigates the effectiveness of the kernel trick in CBMR, specifically, the kernel principal component analysis (KPCA) for dimensionality reduction. KPCA is a nonlinear extension of linear principal component analysis (LPCA) to discovering nonlinear embeddings using the kernel trick. The fundamental idea of KPCA is mapping the input data into a highdimensional feature space through a nonlinear kernel function and then computing the principal components on that mapped space. This paper investigates the potential of KPCA in CBMR for feature extraction or dimensionality reduction. Using the Gaussian kernel in our experiments, we compute the principal components of an image dataset in the transformed space and then we use them as new feature dimensions for the image dataset. Moreover, KPCA can be applied to other many domains including CBMR, where LPCA has been used to extract features and where the nonlinear extension would be effective. Our results from extensive experiments demonstrate that the potential of KPCA is very encouraging compared with LPCA in CBMR.

GC-Tree: A Hierarchical Index Structure for Image Databases (GC-트리 : 이미지 데이타베이스를 위한 계층 색인 구조)

  • 차광호
    • Journal of KIISE:Databases
    • /
    • v.31 no.1
    • /
    • pp.13-22
    • /
    • 2004
  • With the proliferation of multimedia data, there is an increasing need to support the indexing and retrieval of high-dimensional image data. Although there have been many efforts, the performance of existing multidimensional indexing methods is not satisfactory in high dimensions. Thus the dimensionality reduction and the approximate solution methods were tried to deal with the so-called dimensionality curse. But these methods are inevitably accompanied by the loss of precision of query results. Therefore, recently, the vector approximation-based methods such as the VA- file and the LPC-file were developed to preserve the precision of query results. However, the performance of the vector approximation-based methods depend largely on the size of the approximation file and they lose the advantages of the multidimensional indexing methods that prune much search space. In this paper, we propose a new index structure called the GC-tree for efficient similarity search in image databases. The GC-tree is based on a special subspace partitioning strategy which is optimized for clustered high-dimensional images. It adaptively partitions the data space based on a density function and dynamically constructs an index structure. The resultant index structure adapts well to the strongly clustered distribution of high-dimensional images.

Efficient Processing of Multidimensional Sensor stream Data in Digital Marine Vessel (디지털 선박 내 다차원 센서 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Park, Kyung-Woo;Lee, Jin-Seok;Lee, Keong-Hyo;Jung, Min-A;Lee, Sung-Ro
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.5B
    • /
    • pp.794-800
    • /
    • 2010
  • It is necessary to accurate and efficient management for measured digital data from various sensors in digital marine vessel. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. In this paper, We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose that we arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using SVM algorithm. We automatically delete that it isn't necessary to the data from the database and we used to ship diagnosis system for available data. As a result, we obtained to efficient result about 18.3% reduction rate of database using 35,912 data sets.

A Data Centric Storage based on Adaptive Local Trajectory for Sensor Networks (센서네트워크를 위한 적응적 지역 트라젝토리 기반의 데이터 저장소 기법)

  • Lim, Hwa-Jung;Lee, Joa-Hyoung;Yang, Dong-Il;Tscha, Yeong-Hwan;Lee, Heon-Guil
    • The KIPS Transactions:PartC
    • /
    • v.15C no.1
    • /
    • pp.19-30
    • /
    • 2008
  • Sensor nodes are used as a storage space in the data centric storage method for sensor networks. Sensor nodes save the data to the node which is computed by hash table and users also access to the node to get the data by using hash table. One of the problems which the data centric storage method has is that queries from many users who are interested in the popular data could be concentrated to one node. In this case, responses for queries could be delayed and the energy of heavy loaded node could be dissipated fast. This would lead to reduction of network life time. In this paper, ALT, Data Centric Storage based on Adaptive Local Trajectory, is proposed as scalable data centric storage method for sensor network. ALT constructs trajectory around the storage node. The scope of trajectory is increased or decreased based on the query frequency. ALT distributes the query processing loads to several nodes so that delay of response is reduced and energy dissipation is also distributed.