• Title/Summary/Keyword: Data Stream

Search Result 2,505, Processing Time 0.029 seconds

A Spatial Data Stream Processing System for Spatial Context Analysis in Real-time (실시간 공간 상황 분석을 위한 공간 데이터 스트림 처리 시스템)

  • Kwon, O-Je;Kim, Jae-Hun;Li, Ki-Joune
    • Spatial Information Research
    • /
    • v.18 no.1
    • /
    • pp.69-76
    • /
    • 2010
  • Spatial data streams from sensors are useful in context-awareness for many types of applications. However, an important gap is found between spatial data stream management in real-time and complex computation for spatial context-awareness, and this brings about serious difficulty to integrate spatial data stream processing and context-awareness. In this paper, we present a system called SCONSTREAM(Spatial CONtext STREAm Management) that we have developed to resolve the gap between spatial data stream and context-awareness. The key approach of our system is to filter off unnecessary spatial data streams and convert them to the spatial context streams, which are smaller and more suitable to be processed by the context-awareness module than raw data from sensors. By experimentation, We show that SCONSTREAM resolves the functional gap between spatial stream processing and spatial context-awareness module.

A PCA-based Data Stream Reduction Scheme for Sensor Networks (센서 네트워크를 위한 PCA 기반의 데이터 스트림 감소 기법)

  • Fedoseev, Alexander;Choi, Young-Hwan;Hwang, Een-Jun
    • Journal of Internet Computing and Services
    • /
    • v.10 no.4
    • /
    • pp.35-44
    • /
    • 2009
  • The emerging notion of data stream has brought many new challenges to the research communities as a consequence of its conceptual difference with conventional concepts of just data. One typical example is data stream processing in sensor networks. The range of data processing considerations in a sensor network is very wide, from physical resource restrictions such as bandwidth, energy, and memory to the peculiarities of query processing including continuous and specific types of queries. In this paper, as one of the physical constraints in data stream processing, we consider the problem of limited memory and propose a new scheme for data stream reduction based on the Principal Component Analysis (PCA) technique. PCA can transform a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables. We adapt PCA for the data stream of a sensor network assuming the cooperation of a query engine (or application) with a network base station. Our method exploits the spatio-temporal correlation among multiple measurements from different sensors. Finally, we present a new framework for data processing and describe a number of experiments under this framework. We compare our scheme with the wavelet transform and observe the effect of time stamps on the compression ratio. We report on some of the results.

  • PDF

Development of Long Term Flow Duration Curves in 4 River Basins for the Management of Total Maximum Daily Loads (수질오염총량관리를 위한 4대강수계 장기유황곡선 작성방안)

  • Park, Jun Dae;Oh, Seung Young
    • Journal of Korean Society on Water Environment
    • /
    • v.29 no.3
    • /
    • pp.343-353
    • /
    • 2013
  • Flow duration curve (FDC) can be developed by linking the daily flow data of stream flow monitoring network to 8-day interval flow data of the unit watersheds for the management of Total Maximum Daily Loads. This study investigated the applicable method for the development of long term FDC with the selection of the stream flow reference sites, and suggested the development of the FDC in 4 river basins. Out of 142 unit watersheds in 4 river basins, 107 unit watersheds were shown to estimate daily flow data for the unit watersheds from 2006 to 2010. Short term FDC could be developed in 64 unit watersheds (45%) and long term FDC in 43 unit watersheds (30%), while other 35 unit watersheds (25%) were revealed to have difficulties in the development of FDC itself. Limits in the development of the long term FDC includes no stream monitoring sites in certain unit watersheds, short duration of stream flow data set and missing data by abnormal water level measurements on the stream flow monitoring sites. To improve these limits, it is necessary to install new monitoring sites in the required areas, to keep up continuous monitoring and make normal water level observations on the stream flow monitoring sites, and to build up a special management system to enhance data reliability. The development of long term FDC for the unit watersheds can be established appropriately with the normal and durable measurement on the selected reference sites in the stream flow monitoring network.

Boosting Algorithms for Large-Scale Data and Data Batch Stream (대용량 자료와 순차적 자료를 위한 부스팅 알고리즘)

  • Yoon, Young-Joo
    • The Korean Journal of Applied Statistics
    • /
    • v.23 no.1
    • /
    • pp.197-206
    • /
    • 2010
  • In this paper, we propose boosting algorithms when data are very large or coming in batches sequentially over time. In this situation, ordinary boosting algorithm may be inappropriate because it requires the availability of all of the training set at once. To apply to large scale data or data batch stream, we modify the AdaBoost and Arc-x4. These algorithms have good results for both large scale data and data batch stream with or without concept drift on simulated data and real data sets.

A Sliding Window-based Multivariate Stream Data Classification (슬라이딩 윈도우 기반 다변량 스트림 데이타 분류 기법)

  • Seo, Sung-Bo;Kang, Jae-Woo;Nam, Kwang-Woo;Ryu, Keun-Ho
    • Journal of KIISE:Databases
    • /
    • v.33 no.2
    • /
    • pp.163-174
    • /
    • 2006
  • In distributed wireless sensor network, it is difficult to transmit and analyze the entire stream data depending on limited networks, power and processor. Therefore it is suitable to use alternative stream data processing after classifying the continuous stream data. We propose a classification framework for continuous multivariate stream data. The proposed approach works in two steps. In the preprocessing step, it takes input as a sliding window of multivariate stream data and discretizes the data in the window into a string of symbols that characterize the signal changes. In the classification step, it uses a standard text classification algorithm to classify the discretized data in the window. We evaluated both supervised and unsupervised classification algorithms. For supervised, we tested Bayesian classifier and SVM, and for unsupervised, we tested Jaccard, TFIDF Jaro and Jaro Winkler. In our experiments, SVM and TFIDF outperformed other classification methods. In particular, we observed that classification accuracy is improved when the correlation of attributes is also considered along with the n-gram tokens of symbols.

Analysis of Drying Stream Characteristics in the Rural Area (농촌하천 건천화 특성조사 및 분석 -경기 진위천 중심-)

  • Park, Ki-Wook;Yoon, Yeo-Jung;Ju, Uk-Jong
    • Proceedings of the Korean Society of Agricultural Engineers Conference
    • /
    • 2005.10a
    • /
    • pp.68-73
    • /
    • 2005
  • The purpose of this study are to insure flow rate for rural stream in the rural area. The analysis of drying stream characteristics for two streams(Bong-mu, Wang-jang stream) from field survey data. Also, a study area has been determined, and criteria of estimation has been established : criteria such as, available hydrological data, the size of stream lengths and basin areas and regional characteristics. The spacial analysis is applied to stream slopes for upstream and downstream at weirs, stream networks and ground water pump stations, sinuosity of drying stream. As a result of drying streams survey analysis, drying stream characteristics are followed; levee types are earth and natural, cross sectional shapes are trapezoid, stream bed materials are gravels and sands, facilities in streams are weirs. The cause of the reduction analyzed by investigation of the current status of facilities for agricultural water use. Agricultural reservoirs block up the stream and water does not flow over the reservoirs except by storm. They also discharge water through diversion channels and the water diverted does not flow through the natural stream. Farmers directly take water from the stream by weirs.

  • PDF

Development of a Spatio-Temporal DSMS for the Real-time Management of Moving Objects Data Stream (이동체 데이터 스트림의 실시간 관리를 위한 시공간 DSMS의 개발)

  • Shin, In-Su;Kim, Jang-Woo;Kim, Joung-Joon;Han, Ki-Joon
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.20 no.1
    • /
    • pp.21-31
    • /
    • 2012
  • Recently, according to the development of ubiquitous computing technology, the efficient management of locations of moving objects is increasing rapidly in various fields. However, MODBMS and DSMS can not support the efficient real-time management of spatio-temporal stream data of moving objects. Therefore, this paper designed and implemented a spatio-temporal DSMS which can support the efficient real-time management of spatio-temporal stream data of moving objects. Especially, to develop the spatio-temporal DSMS, we extended STREAM of Stanford University and used GEOS that supports spatial data types and spatial operators of OGC. Finally, this paper proved the efficiency of the spatio-temporal DSMS by applying it to the real-time monitoring field which requires the real-time management of spatio-temporal stream data of moving objects.

TIM: A Trapdoor Hash Function-based Authentication Mechanism for Streaming Applications

  • Seo, Seog Chung;Youn, Taek-Young
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.6
    • /
    • pp.2922-2945
    • /
    • 2018
  • Achieving efficient authentication is a crucial issue for stream data commonly seen in content delivery, peer-to-peer, and multicast/broadcast networks. Stream authentication mechanisms need to be operated efficiently at both sender-side and receiver-side at the same time because of the properties of stream data such as real-time and delay-sensitivity. Until now, many stream authentication mechanisms have been proposed, but they are not efficient enough to be used in stream applications where the efficiency for sender and receiver sides are required simultaneously since most of them could achieve one of either sender-side and receiver-side efficiency. In this paper, we propose an efficient stream authentication mechanism, so called TIM, by integrating Trapdoor Hash Function and Merkle Hash Tree. Our construction can support efficient streaming data processing at both sender-side and receiver-side at the same time differently from previously proposed other schemes. Through theoretical and experimental analysis, we show that TIM can provide enhanced performance at both sender and receiver sides compared with existing mechanisms. Furthermore, TIM provides an important feature for streaming authentication, the resilience against transmission loss, since each data block can be verified with authentication information contained in itself.

Estimation of Pollution Load in Anyang Stream Basin Using GIS (GIS를 이용한 안양천 유역의 오염부하량 산정)

  • 최종욱;유병태;이민환;김건흥
    • Journal of environmental and Sanitary engineering
    • /
    • v.14 no.3
    • /
    • pp.1-9
    • /
    • 1999
  • In the estimation of pollution load in water basin, a data information has generally used from surveyed data. A Geographic Information System(GIS) was adopted to evaluate the amount of pollution load in Anyang stream basin which is one of the major tributaries in the Han river flows through urban area. The digital maps of administrative boundary, stream network, sub-basin, soil type, and land-use for spatial data as well as attribute data were generated. And the database of sub-basins and pollution source was structured to estimate pollution load in Anyang stream basin by an Arc/Info GIS.As the results of this investigation, the pollution load of Mokgam-chun sub-basin was the highest amount. And that of Hagi-chun sub-basin and the fourth main stream sub-basin were also high amount in Anyang stream basin. In general, it was found that the pollution load generated from the upstream area in Kyunggi province was higher than that from downstream area in Seoul. Because the point and non-point source pollution load played very significant role in the deterioration of the water quality of the Anyang stream, an integrated approach to water quality management should be required for the sub-basins of high pollution load amount.

  • PDF

Estimation of Sediment Delivery Ratio in Upper Geum River Basin Using Watershed Model (유역모형을 이용한 금강상류 유역의 유사이송율 산정)

  • Kim, Tae Geun;Kim, Min Joo
    • Journal of Environmental Impact Assessment
    • /
    • v.22 no.6
    • /
    • pp.695-703
    • /
    • 2013
  • Soil erosion and sediment delivery ratio(SDR) were estimated by using HSPF model in 3 tributaries of upper stream of Geum river-basin. Meteorological data and other input data were constructed from 2006 to 2011 year by the HSPF model. Flow and suspended solid results were relatively matched with the measurement data through the calibration and validation of the model. Soil erosion was proportional to the amount of rainfall and the area of watershed based on the results of model calibration and validation. SDR in Moojunamdea stream was the highest and one in Cho stream was the lowest. This was effected by the geographical characteristic. SDR was 17.6% Moojunamdea stream, 9.1% Cho stream and 13.2 % Bocheong stream. As the SDR was effected by watershed area and shape factor in this study area.