• Title/Summary/Keyword: Large Volume Data Stream

Search Result 33, Processing Time 0.021 seconds

Real-Time Sensor Monitoring Service based on ECA (ECA 기반 센서 네트워크 실시간 모니터링 서비스)

  • Kim, Jung-Yee
    • Journal of Korea Multimedia Society
    • /
    • v.15 no.1
    • /
    • pp.87-92
    • /
    • 2012
  • Wireless sensor network is a technology that collects the information about object in real-time. Sensor data has a characteristic that is generated an unprecedented volume data in short time. Analysis is essential to define the relationship between the data, including more of the data from a large volume data stream which is acquired from the sensor. In order to effectively handle the sensor data stream, in this paper, using ECA rules to organize data in a meaningful and more practical real-time monitoring systems is proposed.

DESIGN OF A CONTEXT ANALYSIS MODEL ON USN ENVIRONMENT

  • Jin, Cheng-Hao;Lee, Yong-Mi;Nam, Kwang-Woo;Lee, Jun-Wook;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2008.10a
    • /
    • pp.122-125
    • /
    • 2008
  • Sensors used in many USN (Ubiquitous Sensor Network) domain applications generate a large amount of sensor stream data. The volume of sensor stream data is too huge to store the whole data and data speed is too fast to control each of them. In order to provide rapid and reliable context analysis service over sensor stream data, we propose a WHEN-DO context analysis model that supports the functionality of sliding window. This model is designed to be used as follows: If the sensor stream data satisfies condition in 'WHEN' clause, then it will execute actions in 'DO' clause in WHEN-DO context analysis model. The proposed WHEN-DO context analysis model can be applied to many other USN environment applications such as monitoring the status of a building and then taking actions in corresponding context condition.

  • PDF

Stream Data Analysis of the Weather on the Location using Principal Component Analysis (주성분 분석을 이용한 지역기반의 날씨의 스트림 데이터 분석)

  • Kim, Sang-Yeob;Kim, Kwang-Deuk;Bae, Kyoung-Ho;Ryu, Keun-Ho
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.28 no.2
    • /
    • pp.233-237
    • /
    • 2010
  • The recent advance of sensor networks and ubiquitous techniques allow collecting and analyzing of the data which overcome the limitation imposed by time and space in real-time for making decisions. Also, analysis and prediction of collected data can support useful and necessary information to users. The collected data in sensor networks environment is the stream data which has continuous, unlimited and sequential properties. Because of the continuous, unlimited and large volume properties of stream data, managing stream data is difficult. And the stream data needs dynamic processing method because of the memory constraint and access limitation. Accordingly, we analyze correlation stream data using principal component analysis. And using result of analysis, it helps users for making decisions.

A Novel Way of Context-Oriented Data Stream Segmentation using Exon-Intron Theory (Exon-Intron이론을 활용한 상황중심 데이터 스트림 분할 방안)

  • Lee, Seung-Hun;Suh, Dong-Hyok
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.16 no.5
    • /
    • pp.799-806
    • /
    • 2021
  • In the IoT environment, event data from sensors is continuously reported over time. Event data obtained in this trend is accumulated indefinitely, so a method for efficient analysis and management of data is required. In this study, a data stream segmentation method was proposed to support the effective selection and utilization of event data from sensors that are continuously reported and received. An identifier for identifying the point at which to start the analysis process was selected. By introducing the role of these identifiers, it is possible to clarify what is being analyzed and to reduce data throughput. The identifier for stream segmentation proposed in this study is a semantic-oriented data stream segmentation method based on the event occurrence of each stream. The existence of identifiers in stream processing can be said to be useful in terms of providing efficiency and reducing its costs in a large-volume continuous data inflow environment.

Data Source Management using weight table in u-GIS DSMS

  • Kim, Sang-Ki;Baek, Sung-Ha;Lee, Dong-Wook;Chung, Warn-Il;Kim, Gyoung-Bae;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.11 no.2
    • /
    • pp.27-33
    • /
    • 2009
  • The emergences of GeoSensor and researches about GIS have promoted many researches of u-GIS. The disaster application coupled in the u-GIS can apply to monitor accident area and to prevent spread of accident. The application needs the u-GIS DSMS technique to acquire, to process GeoSensor data and to integrate them with GIS data. The u-GIS DSMS must process big and large-volume data stream such as spatial data and multimedia data. Due to the feature of the data stream, in u-GIS DSMS, query processing can be delayed. Moreover, as increasing the input rate of data in the area generating events, the network traffic is increased. To solve this problem, in this paper we describe TRIGGER ACTION clause in CQ on the u-GIS DSMS environment and proposes data source management. Data source weight table controls GES information and incoming data rate. It controls incoming data rate as increasing weight at GES of disaster area. Consequently, it can contribute query processing rate and accuracy

  • PDF

A Review of Window Query Processing for Data Streams

  • Kim, Hyeon Gyu;Kim, Myoung Ho
    • Journal of Computing Science and Engineering
    • /
    • v.7 no.4
    • /
    • pp.220-230
    • /
    • 2013
  • In recent years, progress in hardware technology has resulted in the possibility of monitoring many events in real time. The volume of incoming data may be so large, that monitoring all individual data might be intractable. Revisiting any particular record can also be impossible in this environment. Therefore, many database schemes, such as aggregation, join, frequent pattern mining, and indexing, become more challenging in this context. This paper surveys the previous efforts to resolve these issues in processing data streams. The emphasis is on specifying and processing sliding window queries, which are supported in many stream processing engines. We also review the related work on stream query processing, including synopsis structures, plan sharing, operator scheduling, load shedding, and disorder control.

An Efficient Method for Mining Frequent Patterns based on Weighted Support over Data Streams (데이터 스트림에서 가중치 지지도 기반 빈발 패턴 추출 방법)

  • Kim, Young-Hee;Kim, Won-Young;Kim, Ung-Mo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.10 no.8
    • /
    • pp.1998-2004
    • /
    • 2009
  • Recently, due to technical developments of various storage devices and networks, the amount of data increases rapidly. The large volume of data streams poses unique space and time constraints on the data mining process. The continuous characteristic of streaming data necessitates the use of algorithms that require only one scan over the stream for knowledge discovery. Most of the researches based on the support are concerned with the frequent itemsets, but ignore the infrequent itemsets even if it is crucial. In this paper, we propose an efficient method WSFI-Mine(Weighted Support Frequent Itemsets Mine) to mine all frequent itemsets by one scan from the data stream. This method can discover the closed frequent itemsets using DCT(Data Stream Closed Pattern Tree). We compare the performance of our algorithm with DSM-FI and THUI-Mine, under different minimum supports. As results show that WSFI-Mine not only run significant faster, but also consume less memory.

Derivation of the Instantaneous Unit Hydrograph and Estimation of the Direct Runoff by Using the Geomorphologic Parameters (지상인자에 의한 순간단위도 유도와 유출량 예측)

  • 천만복;서승덕
    • Magazine of the Korean Society of Agricultural Engineers
    • /
    • v.32 no.3
    • /
    • pp.87-101
    • /
    • 1990
  • The purpose of this study is to estimate the flood discharge and runoff volume at a stream by using geomorphologic parameters obtained from the topographic maps following the law of stream classification and ordering by Horton and Strahier. The present model is modified from Cheng' s model which derives the geomorphologic instantaneous unit hydrograph. The present model uses the results of Laplace transformation and convolution intergral of probability density function of the travel time at each state. The stream flow velocity parameters are determined as a function of the rainfall intensity, and the effective rainfall is calculated by the SCS method. The total direct runoff volume until the time to peak is estimated by assuming a triangular hydrograph. The model is used to estimate the time to peak, the flood discharge, and the direct runoff at Andong, Imha. Geomchon, and Sunsan basin in the Nakdong River system. The results of the model application are as follows : 1.For each basin, as the rainfall intensity doubles form 1 mm/h to 2 mm/h with the same rainfall duration of 1 hour, the hydrographs show that the runoff volume doubles while the duration of the base flow and the time to peak are the same. This aggrees with the theory of the unit hydrograph. 2.Comparisions of the model predicted and observed values show that small relative errors of 0.44-7.4% of the flood discharge, and 1 hour difference in time to peak except the Geomchon basin which shows 10.32% and 2 hours respectively. 3.When the rainfall intensity is small, the error of flood discharge estimated by using this model is relatively large. The reason of this might be because of introducing the flood velocity concept in the stream flow velocity. 4.Total direct runoff volume until the time to peak estimated by using this model has small relative error comparing with the observed data. 5.The sensitivity analysis of velocity parameters to flood discharge shows that the flood discharge is sensitive to the velocity coefficient while it is insensitive to the ratio of arrival time of moving portion to that of storage portion of a stream and to the ratio of arrival time of stream to that of overland flow.

  • PDF

Development of 3D Visualization Technology for Meteorological Data (기상자료 3차원 가시화 기술개발 연구)

  • Seo In Bum;Joh Min Su;Yun Ja Young
    • Journal of the Korean Society of Visualization
    • /
    • v.1 no.2
    • /
    • pp.58-70
    • /
    • 2003
  • Meteorological data contains observation and numerical weather prediction model output data. The computerized analysis and visualization of meteorological data often requires very high computing capability due to the large size and complex structure of the data. Because the meteorological data is frequently formed in multi-variables, 3-dimensional and time-series form, it is very important to visualize and analyze the data in 3D spatial domain in order to get more understanding about the meteorological phenomena. In this research, we developed interactive 3-dimensional visualization techniques for visualizing meteorological data on a PC environment such as volume rendering, iso-surface rendering or stream line. The visualization techniques developed in this research are expected to be effectively used as basic technologies not only for deeper understanding and more exact prediction about meteorological environments but also for scientific and spatial data visualization research in any field from which three dimensional data comes out such as oceanography, earth science, and aeronautical engineering.

  • PDF

Streamflow Estimation for Subbasins of Gap Stream Watershed by Using SWAT2000 Model (SWAT2000 모형을 이용한 갑천수계의 소유역별 유출량 추정)

  • Moon, Jong-Pil;Kim, Tai-Cheol
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.48 no.5
    • /
    • pp.29-38
    • /
    • 2006
  • Geographic Information System has extended to higher assessment of water resources. GIS linking with hydrological model becomes a trend in water resource assessment modeling. One of the most popular models is SWAT2000 which have effectiveness in multi-purpose processes for predicting the impact of land management practices on water, sediments and chemicals yields in large complex watershed with varying soils, land uses, and management conditions over long period of time. In this study, SWAT2000 model was applied to Gap stream watershed in Daejeon city where TMDL (Total Maximum Daily Load) Regulation would be implanted. The Gap Stream watershed was partitioned into 8 subbasins, however, only 3 out of 8 subbaisns were observed for having practical gauged data on the basis of streamflow from the year of 2002 to 2005. Gauged streamflow data of Indong, Boksu and Hoeduck stations were used for calibration and validation of the SWAT Streamflow simulation. Estimation Efficiency Analysis (COE), Regression Analysis ($R^{2}$), Relative Error (R.E.) were used for comparing observed streamflow data of the 3 subbasins on the daily and monthly basis with estimated streamflow data in order to fix optimized parameters for the best fitted results. COE value for the daily and monthly streamflow was ranged from 0.45 to 0.96. $R^{2}$ values for daily and monthly streamflow ranged from 0.51 to 0.97. R.E. values for total streamflow volume ranged from 3 % to 22.5 %. The accuracy of the model results shows that the SWAT2000 model can be applicable to Korean watersheds like the Gap Stream watershed that needs to be partitioned into a number of subbasins for TMDL regulation.