• Title/Summary/Keyword: Stream Data

Search Result 2,512, Processing Time 0.035 seconds

Restoration of the Stream Runoff by the Physical Deterministic Modeling and Formulation of Water Balance for the Catchment of Byungchun River in Chungcheong Province in Korea (물리 결정 모델링에 의한 충청도 병천천 유역의 하천 유출량 복원과 물 수지 수립)

  • KIM, Man-Kyu
    • Journal of The Geomorphological Association of Korea
    • /
    • v.15 no.2
    • /
    • pp.37-53
    • /
    • 2008
  • This study has developed a water balance model for the catchment of Byungchun river using a BROOK90 4.4e physical deterministic water balance model with the long-term meterological data and stream run off data obtained from the basin of Byungchun river in Korea. It is intended that the validation model with calibrated model fitting parameter can build a long-term water balance plan for a period when meterological data are available but stream runoff data are not. Results of this study have satisfied the first expectation as an experiment for water balance modeling since measured stream runoff data have turned out to be very similar to simulated stream runoff data. Through the confirmation of model fitting parameters and validated simulation, water balance for the period of 1998 to 2006 has been restored. Unless the conditions of geomophology, vegetation, soil and land use change, meterological data alone can produce various hydrometeorological data related to stream runoff amount, soil water amount, and evapotranspiration. This study opens up a new horizon in restoring water balance in the past as well planning water balance in the present. The obtained results from this study are expected to be used in predicting future water balance in the wake of the changes in climate and vegetation in Korea.

Performance Evaluation and Analysis of Multiple Scenarios of Big Data Stream Computing on Storm Platform

  • Sun, Dawei;Yan, Hongbin;Gao, Shang;Zhou, Zhangbing
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.7
    • /
    • pp.2977-2997
    • /
    • 2018
  • In big data era, fresh data grows rapidly every day. More than 30,000 gigabytes of data are created every second and the rate is accelerating. Many organizations rely heavily on real time streaming, while big data stream computing helps them spot opportunities and risks from real time big data. Storm, one of the most common online stream computing platforms, has been used for big data stream computing, with response time ranging from milliseconds to sub-seconds. The performance of Storm plays a crucial role in different application scenarios, however, few studies were conducted to evaluate the performance of Storm. In this paper, we investigate the performance of Storm under different application scenarios. Our experimental results show that throughput and latency of Storm are greatly affected by the number of instances of each vertex in task topology, and the number of available resources in data center. The fault-tolerant mechanism of Storm works well in most big data stream computing environments. As a result, it is suggested that a dynamic topology, an elastic scheduling framework, and a memory based fault-tolerant mechanism are necessary for providing high throughput and low latency services on Storm platform.

A Method for Frequent Itemsets Mining from Data Stream (데이터 스트림 환경에서 효율적인 빈발 항목 집합 탐사 기법)

  • Seo, Bok-Il;Kim, Jae-In;Hwang, Bu-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.19D no.2
    • /
    • pp.139-146
    • /
    • 2012
  • Data Mining is widely used to discover knowledge in many fields. Although there are many methods to discover association rule, most of them are based on frequency-based approaches. Therefore it is not appropriate for stream environment. Because the stream environment has a property that event data are generated continuously. it is expensive to store all data. In this paper, we propose a new method to discover association rules based on stream environment. Our new method is using a variable window for extracting data items. Variable windows have variable size according to the gap of same target event. Our method extracts data using COBJ(Count object) calculation method. FPMDSTN(Frequent pattern Mining over Data Stream using Terminal Node) discovers association rules from the extracted data items. Through experiment, our method is more efficient to apply stream environment than conventional methods.

A Novel Way of Context-Oriented Data Stream Segmentation using Exon-Intron Theory (Exon-Intron이론을 활용한 상황중심 데이터 스트림 분할 방안)

  • Lee, Seung-Hun;Suh, Dong-Hyok
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.16 no.5
    • /
    • pp.799-806
    • /
    • 2021
  • In the IoT environment, event data from sensors is continuously reported over time. Event data obtained in this trend is accumulated indefinitely, so a method for efficient analysis and management of data is required. In this study, a data stream segmentation method was proposed to support the effective selection and utilization of event data from sensors that are continuously reported and received. An identifier for identifying the point at which to start the analysis process was selected. By introducing the role of these identifiers, it is possible to clarify what is being analyzed and to reduce data throughput. The identifier for stream segmentation proposed in this study is a semantic-oriented data stream segmentation method based on the event occurrence of each stream. The existence of identifiers in stream processing can be said to be useful in terms of providing efficiency and reducing its costs in a large-volume continuous data inflow environment.

Efficient Stream Sequence Matching Algorithms for Handheld Devices over Time-Series Stream Data (시계열 스트림 데이터 상에서 핸드헬드 디바이스를 위한 효율적인 스트림 시퀀스 매칭 알고리즘)

  • Moon Yang-Sae;Loh Woong-Kee
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.31 no.8B
    • /
    • pp.736-744
    • /
    • 2006
  • For the handhold devices, minimizing repetitive CPU operations such as multiplications is a major factor for their performances. In this paper, we propose efficient algorithms for finding similar sequences from streaming time-series data such as stock prices, network traffic data, and sensor network data. First, we formally define the problem of similar subsequence matching from streaming time-series data, which is called the stream sequence matching in this paper. Second, based on the window construction mechanism adopted by the previous subsequence matching algorithms, we present an efficient window-based approach that minimizes CPU operations required for stream sequence matching. Third, we propose a notion of window MBR and present two stream sequence matching algorithms based on the notion. Fourth, we formally prove correctness of the proposed algorithms. Finally, through a series of analyses and experiments, we show that our algorithms significantly outperform the naive algorithm. We believe that our window-based algorithms are excellent choices for embedded stream sequence matching in handhold devices.

DESIGN OF A CONTEXT ANALYSIS MODEL ON USN ENVIRONMENT

  • Jin, Cheng-Hao;Lee, Yong-Mi;Nam, Kwang-Woo;Lee, Jun-Wook;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2008.10a
    • /
    • pp.122-125
    • /
    • 2008
  • Sensors used in many USN (Ubiquitous Sensor Network) domain applications generate a large amount of sensor stream data. The volume of sensor stream data is too huge to store the whole data and data speed is too fast to control each of them. In order to provide rapid and reliable context analysis service over sensor stream data, we propose a WHEN-DO context analysis model that supports the functionality of sliding window. This model is designed to be used as follows: If the sensor stream data satisfies condition in 'WHEN' clause, then it will execute actions in 'DO' clause in WHEN-DO context analysis model. The proposed WHEN-DO context analysis model can be applied to many other USN environment applications such as monitoring the status of a building and then taking actions in corresponding context condition.

  • PDF

열추적자를 이용한 지하수-하천수 혼합대 연구

  • Kim Gu-Yeong;Jeon Cheol-Min;Kim Tae-Hui;Seong Hyeon-Jeong;O Jun-Ho;Kim Yong-Je;Jeong Jae-Hun;Park Seung-Gi
    • Proceedings of the Korean Society of Soil and Groundwater Environment Conference
    • /
    • 2006.04a
    • /
    • pp.277-281
    • /
    • 2006
  • A study on stream-groundwater exchange was performed using head and temperature data of stream water, streambed, and groundwater. Groundwater level and temperature were obtained from multi-depth monitoring wells in small-scale watershed. In the summer time, time series of temperatrue data at streambed and groundwater were monitored for three months. In the winter time, we measured the temperature gradient between stream water and streambed. The observed data showed three typical types of temperature characteristics. First, the temperature of streambed was lower than that of stream water; second, the temperature of streambed and stream water was similar; and last, the temperature of streambed was higher than that of stream water. The interconnections between the stream and the streambed were not homogeneously distributed due to weakly developed sediments and heterogeneous bedrock exposed as bed of the stream. The temperature data may be used in formal solutions of the inverse problems to estimate groundwater flow and hydraulic conductivity.

  • PDF

Classification of large-scale data and data batch stream with forward stagewise algorithm (전진적 단계 알고리즘을 이용한 대용량 데이터와 순차적 배치 데이터의 분류)

  • Yoon, Young Joo
    • Journal of the Korean Data and Information Science Society
    • /
    • v.25 no.6
    • /
    • pp.1283-1291
    • /
    • 2014
  • In this paper, we propose forward stagewise algorithm when data are very large or coming in batches sequentially over time. In this situation, ordinary boosting algorithm for large scale data and data batch stream may be greedy and have worse performance with class noise situations. To overcome those and apply to large scale data or data batch stream, we modify the forward stagewise algorithm. This algorithm has better results for both large scale data and data batch stream with or without concept drift on simulated data and real data sets than boosting algorithms.

WT-Heuristics: An Efficient Filter Operator Ordering Technology in Stream Data Environments (WT-Heuristics: 스트림 데이터 환경에서의 효율적인 필터 연산자 순서화 기법)

  • Min, Jun-Ki
    • The KIPS Transactions:PartD
    • /
    • v.15D no.2
    • /
    • pp.163-170
    • /
    • 2008
  • Due to the proliferation of the Internet and intranet, a new application domain called stream data processing has emerged. Stream data is real-timely and continuously generated. In this paper, we focus on the processing of stream data whose characteristics vary unpredictably by over time. Particularly, we suggest a method which generates an efficient operator execution order called WT-Heuristics. WT-Heuristics efficiently determines the operator execution order since it considers only two adjacent operators in the operator execution order. Also, our method changes the execution order with respect to the change of data characteristics with minimum overheads.

QUISIS: A Query Index Method Using Interval Skip List (QUISIS: Interval Skip List를 활용한 질의 색인 기법)

  • Min, Jun-Ki
    • The KIPS Transactions:PartD
    • /
    • v.15D no.3
    • /
    • pp.297-304
    • /
    • 2008
  • Due to the proliferation of the Internet and intranet, new application domains such as stream data processing have emerged. Stream data is real-timely and continuously generated. In stream data environments, a lot of queries are registered, and then, the arrived data item is evaluated by registered queries. Thus, to accelerate the query performance, diverse continuous query index schemes have been proposed for stream data processing systems. In this paper, we focus on the query index technique for stream data. In general, a stream query contains the range condition. Thus, by using range conditions, the queries can be indexed. In this paper, we propose an efficient query index scheme, called QUISIS, using a modified Interval Skip Lists to accelerate search time. QUISIS utilizes a locality where a value which will arrive in near future is similar to the current value. Through the experimental study, we show the efficiency of our proposed method.