• Title/Summary/Keyword: Spatial Data Stream Processing

Search Result 55, Processing Time 0.021 seconds

A Spatial Data Stream Processing System for Spatial Context Analysis in Real-time (실시간 공간 상황 분석을 위한 공간 데이터 스트림 처리 시스템)

  • Kwon, O-Je;Kim, Jae-Hun;Li, Ki-Joune
    • Spatial Information Research
    • /
    • v.18 no.1
    • /
    • pp.69-76
    • /
    • 2010
  • Spatial data streams from sensors are useful in context-awareness for many types of applications. However, an important gap is found between spatial data stream management in real-time and complex computation for spatial context-awareness, and this brings about serious difficulty to integrate spatial data stream processing and context-awareness. In this paper, we present a system called SCONSTREAM(Spatial CONtext STREAm Management) that we have developed to resolve the gap between spatial data stream and context-awareness. The key approach of our system is to filter off unnecessary spatial data streams and convert them to the spatial context streams, which are smaller and more suitable to be processed by the context-awareness module than raw data from sensors. By experimentation, We show that SCONSTREAM resolves the functional gap between spatial stream processing and spatial context-awareness module.

Spatial Operation Allocation Scheme over Common Query Regions for Distributed Spatial Data Stream Processing (분산 공간 데이터 스트림 처리에서 질의 영역의 겹침을 고려한 공간 연산 배치 기법)

  • Chung, Weon-Il
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.13 no.6
    • /
    • pp.2713-2719
    • /
    • 2012
  • According to increasing of various location-based services, distributed data stream processing techniques have been widely studied to provide high scalability and availability. In previous researches, in order to balance the load of distributed nodes, the geographic characteristics of spatial data stream are not considered. For this reason, distributed operations for adjacent spatial regions increases the overall system load. We propose a operation allocation scheme considering the characteristics of spatial operations to effectively processing spatial data stream in distributed computing environments. The proposed method presents the efficient share maximizing approach that preferentially distributes spatial operations sharing the common query regions to the same node in order to separate the adjacent spatial operations on overlapped regions.

Design and Implementation of Advanced Traffic Monitoring System based on Integration of Data Stream Management System and Spatial DBMS

  • Xia, Ying;Gan, Hongmei;Kim, Gyoung-Bae
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.162-169
    • /
    • 2009
  • The real-time traffic data is generated continuous and unbounded stream data type while intelligent transport system (ITS) needs to provide various and high quality services by combining with spatial information. Traditional database techniques in ITS has shortage for processing dynamic real-time stream data and static spatial data simultaneously. In this paper, we design and implement an advanced traffic monitoring system (ATMS) with the integration of existed data stream management system (DSMS) and spatial DBMS using IntraMap. Besides, the developed ATMS can deal with the stream data of DSMS, the trajectory data of relational DBMS, and the spatial data of SDBMS concurrently. The implemented ATMS supports historical and one time query, continuous query and combined query. Application programmer can develop various intelligent services such as moving trajectory tracking, k-nearest neighbor (KNN) query and dynamic intelligent navigation by using components of the ATMS.

  • PDF

Development of a Spatial DSMS for Efficient Real-Time Processing of Spatial Sensor Data (공간 센서 데이타의 효율적인 실시간 처리를 위한공간 DSMS의 개발)

  • Kang, Hong-Koo;Park, Chi-Min;Hong, Dong-Suk;Han, Ki-Joon
    • Journal of Korea Spatial Information System Society
    • /
    • v.9 no.1
    • /
    • pp.45-57
    • /
    • 2007
  • Recently, the development of sensor devices has accelerated researches on advanced technologies like Wireless Sensor Networks. Moreover, spatial sensors using GPS lead to the era of the Ubiquitous Computing Environment which generally uses spatial information and non-spatial information together. In this new era, a real-time processing system for spatial sensor data is essential. In this reason, new data processing systems called DSMS(Data Stream Management System) are being developed by many researchers. However, since most of them do not support geometry types and spatial functions to process spatial sensor data, they are not suitable for the Ubiquitous Computing Environment. For these reasons, in this paper, we designed and implemented a spatial DSMS by extending STREAM which stands for STanford stREam datA Manager, to solve these problems. We added geometry types and spatial functions to STREAM in order to process spatial sensor data efficiently. In addition, we implemented a Spatial Object Manager to manage shared spatial objects within the system. Especially, we implemented the Simple Features Specification for SQL of OGC for interoperability and applied algorithms in GEOS to our system.

  • PDF

GeoSensor Data Stream Processing System for u-GIS Computing (u-GIS 컴퓨팅을 위한 GeoSensor 데이터 스트림 처리 시스템)

  • Chung, Weon-Il;Shin, Soong-Sun;Back, Sung-Ha;Lee, Yeon;Lee, Dong-Wook;Kim, Kyung-Bae;Lee, Chung-Ho;Kim, Ju-Wan;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.1
    • /
    • pp.9-16
    • /
    • 2009
  • In ubiquitous spatial computing environments, GeoSensor generates sensor data streams including spatial information as well as various conventional sensor data from RFID, WSN, Web CAM, Digital Camera, CCTV, and Telematics units. This GeoSensor enables the revitalization of various ubiquitous USN technologies and services on geographic information. In order to service the u-GIS applications based on GeoSensors, it is indispensable to efficiently process sensor data streams from GeoSensors of a wide area. In this paper, we propose a GeoSensor data stream processing system for u-GIS computing over real-time stream data from GeoSensors with geographic information. The proposed system provides efficient gathering, storing, and continuous query processing of GeoSensor data stream, and also makes it possible to develop diverse u-GIS applications meet each user requirements effectively.

  • PDF

Load Balancing for Distributed Processing of Real-time Spatial Big Data Stream (실시간 공간 빅데이터 스트림 분산 처리를 위한 부하 균형화 방법)

  • Yoon, Susik;Lee, Jae-Gil
    • Journal of KIISE
    • /
    • v.44 no.11
    • /
    • pp.1209-1218
    • /
    • 2017
  • A variety of sensors is widely used these days, and it has become much easier to acquire spatial big data streams from various sources. Since spatial data streams have inherently skewed and dynamically changing distributions, the system must effectively distribute the load among workers. Previous studies to solve this load imbalance problem are not directly applicable to processing spatial data. In this research, we propose Adaptive Spatial Key Grouping (ASKG). The main idea of ASKG is, by utilizing the previous distribution of the data streams, to adaptively suggest a new grouping scheme that evenly distributes the future load among workers. We evaluate the validity of the proposed algorithm in various environments, by conducting an experiment with real datasets while varying the number of workers, input rate, and processing overhead. Compared to two other alternative algorithms, ASKG improves the system performance in terms of load imbalance, throughput, and latency.

Dynamic Load Management Method for Spatial Data Stream Processing on MapReduce Online Frameworks (맵리듀스 온라인 프레임워크에서 공간 데이터 스트림 처리를 위한 동적 부하 관리 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.8
    • /
    • pp.535-544
    • /
    • 2018
  • As the spread of mobile devices equipped with various sensors and high-quality wireless network communications functionsexpands, the amount of spatio-temporal data generated from mobile devices in various service fields is rapidly increasing. In conventional research into processing a large amount of real-time spatio-temporal streams, it is very difficult to apply a Hadoop-based spatial big data system, designed to be a batch processing platform, to a real-time service for spatio-temporal data streams. This paper extends the MapReduce online framework to support real-time query processing for continuous-input, spatio-temporal data streams, and proposes a load management method to distribute overloads for efficient query processing. The proposed scheme shows a dynamic load balancing method for the nodes based on the inflow rate and the load factor of the input data based on the space partition. Experiments show that it is possible to support efficient query processing by distributing the spatial data stream in the corresponding area to the shared resources when load management in a specific area is required.

Design and Implementation of a USN Middleware for Context-Aware and Sensor Stream Mining

  • Jin, Cheng-Hao;Lee, Yang-Koo;Lee, Seong-Ho;Yun, Un-il;Ryu, Keun-Ho
    • Spatial Information Research
    • /
    • v.19 no.1
    • /
    • pp.127-133
    • /
    • 2011
  • Recently, with the advances in sensor techniques and net work computing, Ubiquitous Sensor Network (USN) has been received a lot of attentions from various communities. The sensor nodes distributed in the sensor network tend to continuously generate a large amount of data, which is called stream data. Sensor stream data arrives in an online manner so that it is characterized as high-speed, real-time and unbounded and it requires fast data processing to get the up-to-date results. The data stream has many application domains such as traffic analysis, physical distribution, U-healthcare and so on. Therefore, there is an overwhelming need of a USN middleware for processing such online stream data to provide corresponding services to diverse applications. In this paper, we propose a novel USN middleware which can provide users both context-aware service and meaningful sequential patterns. Our proposed USN middleware is mainly focused on location based applications which use stream location data. We also show the implementation of our proposed USN middleware. By using the proposed USN middleware, we can save the developing cost of providing context aware services and stream sequential patterns mainly in location based applications.

Adaptive Upstream Backup Scheme based on Throughput Rate in Distributed Spatial Data Stream System (분산 공간 데이터 스트림 시스템에서 연산 처리율 기반의 적응적 업스트림 백업 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.10
    • /
    • pp.5156-5161
    • /
    • 2013
  • In distributed spatial data stream processing, processed tuples of downstream nodes are replicated to the upstream node in order to increase the utilization of distributed nodes and to recover the whole system for the case of system failure. However, while the data input rate increases and multiple downstream nodes share the operation result of the upstream node, the data which stores to output queues as a backup can be lost since the deletion operation delay may be occurred by the delay of the tuple processing of upstream node. In this paper, the adaptive upstream backup scheme based on operation throughput in distributed spatial data stream system is proposed. This method can cut down the average load rate of nodes by efficient spatial operation migration as it processes spatial temporal data stream, and it can minimize the data loss by fluid change of backup mode. The experiments show the proposed approach can prevent data loss and can decrease, on average, 20% of CPU utilization by node monitoring.

Partition-based Operator Sharing Scheme for Spatio-temporal Data Stream Processing (시공간 데이터 스트림 처리를 위한 영역 기반의 연산자 공유 기법)

  • Chung, Weon-Il;Kim, Young-Ki
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.11 no.12
    • /
    • pp.5042-5048
    • /
    • 2010
  • In ubiquitous environments, many continuous query processing techniques make use of operator network and sharing methods on continuous data stream generated from various sensors. Since similar continuous queries with the location information intensively occur in specific regions, we suggest a new operator sharing method based on grid partition for the spatial continuous query processing for location-based applications. Due to the proposed method shares moving objects by the given grid cell without sharing spatial operators individually, our approach can not only share spatial operators including similar conditions, but also increase the query processing performance and the utilization of memory by reducing the frequency of use of spatial operators.