• 제목/요약/키워드: Large Volume Data Stream

검색결과 33건 처리시간 0.024초

ECA 기반 센서 네트워크 실시간 모니터링 서비스 (Real-Time Sensor Monitoring Service based on ECA)

  • 김정이
    • 한국멀티미디어학회논문지
    • /
    • 제15권1호
    • /
    • pp.87-92
    • /
    • 2012
  • 무선 센서 네트워크는 실시간으로 객체에 대한 정보를 수집하기 위한 기술로 짧은 시간에 많은 양의 데이터가 지속적으로 발생되는 특성을 갖고 있다. 센서로부터 획득되는 많은 양의 데이터 스트림을 효과적으로 처리하기 위해서는 데이터 간의 관계를 정의하여 분석, 사용자가 주목할 만한 상황, 즉 센서 데이터들 중 이상 데이터가 포함되어 있는 것을 발견하는 것이 필수적이라 할 수 있다. 본 논문에서는 센서 데이터 스트림을 효과적으로 처리하기 위해 ECA 규칙을 사용하여 의미있는 데이터를 구성하고, 이를 통해 보다 실용적인 실시간 모니터링 시스템을 제안한다.

DESIGN OF A CONTEXT ANALYSIS MODEL ON USN ENVIRONMENT

  • Jin, Cheng-Hao;Lee, Yong-Mi;Nam, Kwang-Woo;Lee, Jun-Wook;Ryu, Keun-Ho
    • 대한원격탐사학회:학술대회논문집
    • /
    • 대한원격탐사학회 2008년도 International Symposium on Remote Sensing
    • /
    • pp.122-125
    • /
    • 2008
  • Sensors used in many USN (Ubiquitous Sensor Network) domain applications generate a large amount of sensor stream data. The volume of sensor stream data is too huge to store the whole data and data speed is too fast to control each of them. In order to provide rapid and reliable context analysis service over sensor stream data, we propose a WHEN-DO context analysis model that supports the functionality of sliding window. This model is designed to be used as follows: If the sensor stream data satisfies condition in 'WHEN' clause, then it will execute actions in 'DO' clause in WHEN-DO context analysis model. The proposed WHEN-DO context analysis model can be applied to many other USN environment applications such as monitoring the status of a building and then taking actions in corresponding context condition.

  • PDF

주성분 분석을 이용한 지역기반의 날씨의 스트림 데이터 분석 (Stream Data Analysis of the Weather on the Location using Principal Component Analysis)

  • 김상엽;김광덕;배경호;류근호
    • 한국측량학회지
    • /
    • 제28권2호
    • /
    • pp.233-237
    • /
    • 2010
  • The recent advance of sensor networks and ubiquitous techniques allow collecting and analyzing of the data which overcome the limitation imposed by time and space in real-time for making decisions. Also, analysis and prediction of collected data can support useful and necessary information to users. The collected data in sensor networks environment is the stream data which has continuous, unlimited and sequential properties. Because of the continuous, unlimited and large volume properties of stream data, managing stream data is difficult. And the stream data needs dynamic processing method because of the memory constraint and access limitation. Accordingly, we analyze correlation stream data using principal component analysis. And using result of analysis, it helps users for making decisions.

Exon-Intron이론을 활용한 상황중심 데이터 스트림 분할 방안 (A Novel Way of Context-Oriented Data Stream Segmentation using Exon-Intron Theory)

  • 이승훈;서동혁
    • 한국전자통신학회논문지
    • /
    • 제16권5호
    • /
    • pp.799-806
    • /
    • 2021
  • 사물인터넷 환경에서는 센서로부터의 이벤트 데이터가 시간의 흐름에 따라 지속적으로 보고된다. 이러한 추세로 입수되는 이벤트 데이터는 무한정 쌓이게 되므로 데이터의 효율적인 분석과 관리를 위한 방안이 필요하다. 본 연구에서는 지속적으로 보고되어 유입되는 센서로부터의 이벤트 데이터에 대하여 효과적인 선택과 활용을 뒷받침 할 수 있도록 하는 데이터 스트림 분할 방안을 제안하였다. 분석 처리를 시작할 지점을 식별하기 위한 식별자를 선정하도록 하였다. 이러한 식별자의 역할을 존치시킴으로써 분석할 대상을 명확하게 할 수 있으며 데이터 처리량을 감소시킬 수 있다. 본 연구에서 제안하는 스트림 분할을 위한 식별자는 각 스트림의 이벤트 발생을 기준으로 하기에 의미 중심의 데이터 스트림 분할 방안이라고 할 수 있다. 스트림 처리에서의 식별자의 존재는 대용량의 지속적인 데이터 유입환경에서 효율성을 제공하고 비용을 저감하는 측면에서 유용하다고 할 수 있다.

Data Source Management using weight table in u-GIS DSMS

  • Kim, Sang-Ki;Baek, Sung-Ha;Lee, Dong-Wook;Chung, Warn-Il;Kim, Gyoung-Bae;Bae, Hae-Young
    • 한국공간정보시스템학회 논문지
    • /
    • 제11권2호
    • /
    • pp.27-33
    • /
    • 2009
  • The emergences of GeoSensor and researches about GIS have promoted many researches of u-GIS. The disaster application coupled in the u-GIS can apply to monitor accident area and to prevent spread of accident. The application needs the u-GIS DSMS technique to acquire, to process GeoSensor data and to integrate them with GIS data. The u-GIS DSMS must process big and large-volume data stream such as spatial data and multimedia data. Due to the feature of the data stream, in u-GIS DSMS, query processing can be delayed. Moreover, as increasing the input rate of data in the area generating events, the network traffic is increased. To solve this problem, in this paper we describe TRIGGER ACTION clause in CQ on the u-GIS DSMS environment and proposes data source management. Data source weight table controls GES information and incoming data rate. It controls incoming data rate as increasing weight at GES of disaster area. Consequently, it can contribute query processing rate and accuracy

  • PDF

A Review of Window Query Processing for Data Streams

  • Kim, Hyeon Gyu;Kim, Myoung Ho
    • Journal of Computing Science and Engineering
    • /
    • 제7권4호
    • /
    • pp.220-230
    • /
    • 2013
  • In recent years, progress in hardware technology has resulted in the possibility of monitoring many events in real time. The volume of incoming data may be so large, that monitoring all individual data might be intractable. Revisiting any particular record can also be impossible in this environment. Therefore, many database schemes, such as aggregation, join, frequent pattern mining, and indexing, become more challenging in this context. This paper surveys the previous efforts to resolve these issues in processing data streams. The emphasis is on specifying and processing sliding window queries, which are supported in many stream processing engines. We also review the related work on stream query processing, including synopsis structures, plan sharing, operator scheduling, load shedding, and disorder control.

데이터 스트림에서 가중치 지지도 기반 빈발 패턴 추출 방법 (An Efficient Method for Mining Frequent Patterns based on Weighted Support over Data Streams)

  • 김영희;김원영;김응모
    • 한국산학기술학회논문지
    • /
    • 제10권8호
    • /
    • pp.1998-2004
    • /
    • 2009
  • 다양한 저장 장치의 발달과 네트워크의 발전은 대용량의 데이터를 연속적으로 빠르게 생성한다. 데이터 스트림에서의 데이터 마이닝은 처리 시간 및 메모리 사용에 제한적이다. 또한 생성된 데이터를 한 번의 스캔으로 유용한 패턴을 발견할 수 있어야 하고 정보 변화 가능성이 큰 데이터 속성을 갖는 경우 최근의 정보를 반영한 빠른 분석이 가능해야 한다. 기존의 지지도 기반 마이닝 방법들은 일정 기간 동안 미리 정의된 지지도 이상의 빈발 항목에 대하여만 고려하므로 중요도가 높은 항목들을 간과하는 문제점을 가지고 있다. 본 논문에서는 시간의 변화에 따른 가변성을 고려하여 가중치 지지도를 갖는 데이터 항목들에 대하여 보다 의미 있는 정보를 제공하기 위한 효율적인 빈발패턴 추출 방법을 제안하고자 한다. 제안된 WSFI-Mine(Weighted Support Frequent Itemsets Mine) 방법은 DCT(Data Stream Closed Pattern Tree) 데이터 구조를 이용하여 패쇄 빈발 항목을 탐사한다. 제안된 알고리즘은 DSM-FI와 THUI-Mine 알고리즘과 지지도 변화에 따른 성능을 비교하였고 그 결과 비교 알고리즘 보다 수행 시간이 우수함을 보였고, 빈발 항목을 생성하는 후보 항목의 수를 줄이므로 메모리 사용량을 효율적으로 사용할 수 있음을 보였다.

지상인자에 의한 순간단위도 유도와 유출량 예측 (Derivation of the Instantaneous Unit Hydrograph and Estimation of the Direct Runoff by Using the Geomorphologic Parameters)

  • 천만복;서승덕
    • 한국농공학회지
    • /
    • 제32권3호
    • /
    • pp.87-101
    • /
    • 1990
  • The purpose of this study is to estimate the flood discharge and runoff volume at a stream by using geomorphologic parameters obtained from the topographic maps following the law of stream classification and ordering by Horton and Strahier. The present model is modified from Cheng' s model which derives the geomorphologic instantaneous unit hydrograph. The present model uses the results of Laplace transformation and convolution intergral of probability density function of the travel time at each state. The stream flow velocity parameters are determined as a function of the rainfall intensity, and the effective rainfall is calculated by the SCS method. The total direct runoff volume until the time to peak is estimated by assuming a triangular hydrograph. The model is used to estimate the time to peak, the flood discharge, and the direct runoff at Andong, Imha. Geomchon, and Sunsan basin in the Nakdong River system. The results of the model application are as follows : 1.For each basin, as the rainfall intensity doubles form 1 mm/h to 2 mm/h with the same rainfall duration of 1 hour, the hydrographs show that the runoff volume doubles while the duration of the base flow and the time to peak are the same. This aggrees with the theory of the unit hydrograph. 2.Comparisions of the model predicted and observed values show that small relative errors of 0.44-7.4% of the flood discharge, and 1 hour difference in time to peak except the Geomchon basin which shows 10.32% and 2 hours respectively. 3.When the rainfall intensity is small, the error of flood discharge estimated by using this model is relatively large. The reason of this might be because of introducing the flood velocity concept in the stream flow velocity. 4.Total direct runoff volume until the time to peak estimated by using this model has small relative error comparing with the observed data. 5.The sensitivity analysis of velocity parameters to flood discharge shows that the flood discharge is sensitive to the velocity coefficient while it is insensitive to the ratio of arrival time of moving portion to that of storage portion of a stream and to the ratio of arrival time of stream to that of overland flow.

  • PDF

기상자료 3차원 가시화 기술개발 연구 (Development of 3D Visualization Technology for Meteorological Data)

  • 서인범;조민수;윤자영
    • 한국가시화정보학회지
    • /
    • 제1권2호
    • /
    • pp.58-70
    • /
    • 2003
  • Meteorological data contains observation and numerical weather prediction model output data. The computerized analysis and visualization of meteorological data often requires very high computing capability due to the large size and complex structure of the data. Because the meteorological data is frequently formed in multi-variables, 3-dimensional and time-series form, it is very important to visualize and analyze the data in 3D spatial domain in order to get more understanding about the meteorological phenomena. In this research, we developed interactive 3-dimensional visualization techniques for visualizing meteorological data on a PC environment such as volume rendering, iso-surface rendering or stream line. The visualization techniques developed in this research are expected to be effectively used as basic technologies not only for deeper understanding and more exact prediction about meteorological environments but also for scientific and spatial data visualization research in any field from which three dimensional data comes out such as oceanography, earth science, and aeronautical engineering.

  • PDF

SWAT2000 모형을 이용한 갑천수계의 소유역별 유출량 추정 (Streamflow Estimation for Subbasins of Gap Stream Watershed by Using SWAT2000 Model)

  • 문종필;김태철
    • 한국농공학회논문집
    • /
    • 제48권5호
    • /
    • pp.29-38
    • /
    • 2006
  • Geographic Information System has extended to higher assessment of water resources. GIS linking with hydrological model becomes a trend in water resource assessment modeling. One of the most popular models is SWAT2000 which have effectiveness in multi-purpose processes for predicting the impact of land management practices on water, sediments and chemicals yields in large complex watershed with varying soils, land uses, and management conditions over long period of time. In this study, SWAT2000 model was applied to Gap stream watershed in Daejeon city where TMDL (Total Maximum Daily Load) Regulation would be implanted. The Gap Stream watershed was partitioned into 8 subbasins, however, only 3 out of 8 subbaisns were observed for having practical gauged data on the basis of streamflow from the year of 2002 to 2005. Gauged streamflow data of Indong, Boksu and Hoeduck stations were used for calibration and validation of the SWAT Streamflow simulation. Estimation Efficiency Analysis (COE), Regression Analysis ($R^{2}$), Relative Error (R.E.) were used for comparing observed streamflow data of the 3 subbasins on the daily and monthly basis with estimated streamflow data in order to fix optimized parameters for the best fitted results. COE value for the daily and monthly streamflow was ranged from 0.45 to 0.96. $R^{2}$ values for daily and monthly streamflow ranged from 0.51 to 0.97. R.E. values for total streamflow volume ranged from 3 % to 22.5 %. The accuracy of the model results shows that the SWAT2000 model can be applicable to Korean watersheds like the Gap Stream watershed that needs to be partitioned into a number of subbasins for TMDL regulation.