• Title/Summary/Keyword: DataStream Analysis

Search Result 903, Processing Time 0.028 seconds

Applying Formal Methods to Modeling and Analysis of Real-time Data Streams

  • Kapitanova, Krasimira;Wei, Yuan;Kang, Woo-Chul;Son, Sang-H.
    • Journal of Computing Science and Engineering
    • /
    • v.5 no.1
    • /
    • pp.85-110
    • /
    • 2011
  • Achieving situation awareness is especially challenging for real-time data stream applications because they i) operate on continuous unbounded streams of data, and ii) have inherent realtime requirements. In this paper we showed how formal data stream modeling and analysis can be used to better understand stream behavior, evaluate query costs, and improve application performance. We used MEDAL, a formal specification language based on Petri nets, to model the data stream queries and the quality-of-service management mechanisms of RT-STREAM, a prototype system for data stream management. MEDAL's ability to combine query logic and data admission control in one model allows us to design a single comprehensive model of the system. This model can be used to perform a large set of analyses to help improve the application's performance and quality of service.

Stream Data Analysis of the Weather on the Location using Principal Component Analysis (주성분 분석을 이용한 지역기반의 날씨의 스트림 데이터 분석)

  • Kim, Sang-Yeob;Kim, Kwang-Deuk;Bae, Kyoung-Ho;Ryu, Keun-Ho
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.28 no.2
    • /
    • pp.233-237
    • /
    • 2010
  • The recent advance of sensor networks and ubiquitous techniques allow collecting and analyzing of the data which overcome the limitation imposed by time and space in real-time for making decisions. Also, analysis and prediction of collected data can support useful and necessary information to users. The collected data in sensor networks environment is the stream data which has continuous, unlimited and sequential properties. Because of the continuous, unlimited and large volume properties of stream data, managing stream data is difficult. And the stream data needs dynamic processing method because of the memory constraint and access limitation. Accordingly, we analyze correlation stream data using principal component analysis. And using result of analysis, it helps users for making decisions.

DESIGN OF A CONTEXT ANALYSIS MODEL ON USN ENVIRONMENT

  • Jin, Cheng-Hao;Lee, Yong-Mi;Nam, Kwang-Woo;Lee, Jun-Wook;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2008.10a
    • /
    • pp.122-125
    • /
    • 2008
  • Sensors used in many USN (Ubiquitous Sensor Network) domain applications generate a large amount of sensor stream data. The volume of sensor stream data is too huge to store the whole data and data speed is too fast to control each of them. In order to provide rapid and reliable context analysis service over sensor stream data, we propose a WHEN-DO context analysis model that supports the functionality of sliding window. This model is designed to be used as follows: If the sensor stream data satisfies condition in 'WHEN' clause, then it will execute actions in 'DO' clause in WHEN-DO context analysis model. The proposed WHEN-DO context analysis model can be applied to many other USN environment applications such as monitoring the status of a building and then taking actions in corresponding context condition.

  • PDF

Analysis of Drying Stream Characteristics in the Rural Area (농촌하천 건천화 특성조사 및 분석 -경기 진위천 중심-)

  • Park, Ki-Wook;Yoon, Yeo-Jung;Ju, Uk-Jong
    • Proceedings of the Korean Society of Agricultural Engineers Conference
    • /
    • 2005.10a
    • /
    • pp.68-73
    • /
    • 2005
  • The purpose of this study are to insure flow rate for rural stream in the rural area. The analysis of drying stream characteristics for two streams(Bong-mu, Wang-jang stream) from field survey data. Also, a study area has been determined, and criteria of estimation has been established : criteria such as, available hydrological data, the size of stream lengths and basin areas and regional characteristics. The spacial analysis is applied to stream slopes for upstream and downstream at weirs, stream networks and ground water pump stations, sinuosity of drying stream. As a result of drying streams survey analysis, drying stream characteristics are followed; levee types are earth and natural, cross sectional shapes are trapezoid, stream bed materials are gravels and sands, facilities in streams are weirs. The cause of the reduction analyzed by investigation of the current status of facilities for agricultural water use. Agricultural reservoirs block up the stream and water does not flow over the reservoirs except by storm. They also discharge water through diversion channels and the water diverted does not flow through the natural stream. Farmers directly take water from the stream by weirs.

  • PDF

DPICM subprojectile counting technique using image analysis of infrared camera (적외선 영상해석을 이용한 이중목적탄 자탄계수 계측기법연구)

  • Park, Won-Woo;Choi, Ju-Ho;Lyou, Joon
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1997.10a
    • /
    • pp.11-16
    • /
    • 1997
  • This paper describes the grenade counting system developed for DPICM submunition analysis using the infrared video streams, and its some video stream processing technique. The video stream data processing procedure consists of four sequences; Analog infrared video stream recording, video stream capture, video stream pre-processing, and video stream analysis including the grenade counting. Some applications of this algorithms to real bursting test has shown the possibility of automation for submunition counting.

  • PDF

Multi-dimensional Query Authentication for On-line Stream Analytics

  • Chen, Xiangrui;Kim, Gyoung-Bae;Bae, Hae-Young
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.4 no.2
    • /
    • pp.154-173
    • /
    • 2010
  • Database outsourcing is unavoidable in the near future. In the scenario of data stream outsourcing, the data owner continuously publishes the latest data and associated authentication information through a service provider. Clients may register queries to the service provider and verify the result's correctness, utilizing the additional authentication information. Research on On-line Stream Analytics (OLSA) is motivated by extending the data cube technology for higher multi-level abstraction on the low-level-abstracted data streams. Existing work on OLSA fails to consider the issue of database outsourcing, while previous work on stream authentication does not support OLSA. To close this gap and solve the problem of OLSA query authentication while outsourcing data streams, we propose MDAHRB and MDAHB, two multi-dimensional authentication approaches. They are based on the general data model for OLSA, the stream cube. First, we improve the data structure of the H-tree, which is used to store the stream cube. Then, we design and implement two authentication schemes based on the improved H-trees, the HRB- and HB-trees, in accordance with the main stream query authentication framework for database outsourcing. Along with a cost models analysis, consistent with state-of-the-art cost metrics, an experimental evaluation is performed on a real data set. It exhibits that both MDAHRB and MDAHB are feasible for authenticating OLSA queries, while MDAHRB is more scalable.

Relationship between the Flow data on the Unit Watersheds and on the Stream Flow Monitoring Network (수질오염총량관리 단위유역 유량자료와 하천유량 측정망 자료의 연계성 분석)

  • Park, Jun Dae;Oh, Seung Young
    • Journal of Korean Society on Water Environment
    • /
    • v.29 no.1
    • /
    • pp.55-65
    • /
    • 2013
  • It is very difficult to apply stream flow data directly to the management of Total Maximum Daily Loads because there are some differences between the unit watershed and the stream flow monitoring network in their characteristics such as monitoring locations and its intervals. Flow duration curve can be developed by linking the daily flow data of stream monitoring network to 8 day interval flow data of the unit watershed. This study investigated the current operating conditions of the stream flow monitoring network and the flow relationships between the unit watershed and the stream flow monitoring network. Criteria such as missing and zero value data, and correlation coefficients were applied to select the stream flow reference sites. The reference sites were selected in 112 areas out of 142 unit watersheds in 4 river basins, where the stream flow observations were carried out in relatively normal operating conditions. These reference sites could be utilized in various ways such as flow variation analysis, flow duration curve development and so on for the management of Total Maximum Daily Loads.

A Novel Way of Context-Oriented Data Stream Segmentation using Exon-Intron Theory (Exon-Intron이론을 활용한 상황중심 데이터 스트림 분할 방안)

  • Lee, Seung-Hun;Suh, Dong-Hyok
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.16 no.5
    • /
    • pp.799-806
    • /
    • 2021
  • In the IoT environment, event data from sensors is continuously reported over time. Event data obtained in this trend is accumulated indefinitely, so a method for efficient analysis and management of data is required. In this study, a data stream segmentation method was proposed to support the effective selection and utilization of event data from sensors that are continuously reported and received. An identifier for identifying the point at which to start the analysis process was selected. By introducing the role of these identifiers, it is possible to clarify what is being analyzed and to reduce data throughput. The identifier for stream segmentation proposed in this study is a semantic-oriented data stream segmentation method based on the event occurrence of each stream. The existence of identifiers in stream processing can be said to be useful in terms of providing efficiency and reducing its costs in a large-volume continuous data inflow environment.

An Expanded Patching Technique using Four Types of Streams for True VoD Services

  • Ha, Sook-Jeong;Bae, Ihn-Han;Kim, Jin-Gyu;Park, Young-Ho;Oh, Sun-Jin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.3 no.5
    • /
    • pp.444-460
    • /
    • 2009
  • In this paper, we propose an expanded patching technique in order to reduce the server network bandwidth requirements to support true VoD services in VoD Systems. Double Patching, which is a typical multicast technique, ensures that a long patching stream delivers not only essential video data for the current client but also extra video data for future clients. Since the extra data may include useless data, it results in server network bandwidth wastage. In order to prevent a server from transmitting useless data, the proposed patching technique uses a new kind of stream called a linking stream. A linking stream is transmitted to clients that have received short patching streams, and it plays a linking role between a patching stream and a regular stream. The linking stream enables a server to avoid transmitting unnecessary data delivered by a long patching stream in Double Patching, so the server never wastes its network bandwidth. Mathematical analysis shows that the proposed technique requires less server network bandwidth to support true VoD services than Double Patching. Moreover, simulation results show that it has better average service latency and client defection rate compared with Double Patching.

Design and Implementation of a USN Middleware for Context-Aware and Sensor Stream Mining

  • Jin, Cheng-Hao;Lee, Yang-Koo;Lee, Seong-Ho;Yun, Un-il;Ryu, Keun-Ho
    • Spatial Information Research
    • /
    • v.19 no.1
    • /
    • pp.127-133
    • /
    • 2011
  • Recently, with the advances in sensor techniques and net work computing, Ubiquitous Sensor Network (USN) has been received a lot of attentions from various communities. The sensor nodes distributed in the sensor network tend to continuously generate a large amount of data, which is called stream data. Sensor stream data arrives in an online manner so that it is characterized as high-speed, real-time and unbounded and it requires fast data processing to get the up-to-date results. The data stream has many application domains such as traffic analysis, physical distribution, U-healthcare and so on. Therefore, there is an overwhelming need of a USN middleware for processing such online stream data to provide corresponding services to diverse applications. In this paper, we propose a novel USN middleware which can provide users both context-aware service and meaningful sequential patterns. Our proposed USN middleware is mainly focused on location based applications which use stream location data. We also show the implementation of our proposed USN middleware. By using the proposed USN middleware, we can save the developing cost of providing context aware services and stream sequential patterns mainly in location based applications.