• Title/Summary/Keyword: Data Stream

Search Result 2,505, Processing Time 0.028 seconds

Design and Implementation of a USN Middleware for Context-Aware and Sensor Stream Mining

  • Jin, Cheng-Hao;Lee, Yang-Koo;Lee, Seong-Ho;Yun, Un-il;Ryu, Keun-Ho
    • Spatial Information Research
    • /
    • v.19 no.1
    • /
    • pp.127-133
    • /
    • 2011
  • Recently, with the advances in sensor techniques and net work computing, Ubiquitous Sensor Network (USN) has been received a lot of attentions from various communities. The sensor nodes distributed in the sensor network tend to continuously generate a large amount of data, which is called stream data. Sensor stream data arrives in an online manner so that it is characterized as high-speed, real-time and unbounded and it requires fast data processing to get the up-to-date results. The data stream has many application domains such as traffic analysis, physical distribution, U-healthcare and so on. Therefore, there is an overwhelming need of a USN middleware for processing such online stream data to provide corresponding services to diverse applications. In this paper, we propose a novel USN middleware which can provide users both context-aware service and meaningful sequential patterns. Our proposed USN middleware is mainly focused on location based applications which use stream location data. We also show the implementation of our proposed USN middleware. By using the proposed USN middleware, we can save the developing cost of providing context aware services and stream sequential patterns mainly in location based applications.

Frequent Items Mining based on Regression Model in Data Streams (스트림 데이터에서 회귀분석에 기반한 빈발항목 예측)

  • Lee, Uk-Hyun
    • The Journal of the Korea Contents Association
    • /
    • v.9 no.1
    • /
    • pp.147-158
    • /
    • 2009
  • Recently, the data model in stream data environment has massive, continuous, and infinity properties. However the stream data processing like query process or data analysis is conducted using a limited capacity of disk or memory. In these environment, the traditional frequent pattern discovery on transaction database can be performed because it is difficult to manage the information continuously whether a continuous stream data is the frequent item or not. In this paper, we propose the method which we are able to predict the frequent items using the regression model on continuous stream data environment. We can use as a prediction model on indefinite items by constructing the regression model on stream data. We will show that the proposed method is able to be efficiently used on stream data environment through a variety of experiments.

Stream Data Analysis of the Weather on the Location using Principal Component Analysis (주성분 분석을 이용한 지역기반의 날씨의 스트림 데이터 분석)

  • Kim, Sang-Yeob;Kim, Kwang-Deuk;Bae, Kyoung-Ho;Ryu, Keun-Ho
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.28 no.2
    • /
    • pp.233-237
    • /
    • 2010
  • The recent advance of sensor networks and ubiquitous techniques allow collecting and analyzing of the data which overcome the limitation imposed by time and space in real-time for making decisions. Also, analysis and prediction of collected data can support useful and necessary information to users. The collected data in sensor networks environment is the stream data which has continuous, unlimited and sequential properties. Because of the continuous, unlimited and large volume properties of stream data, managing stream data is difficult. And the stream data needs dynamic processing method because of the memory constraint and access limitation. Accordingly, we analyze correlation stream data using principal component analysis. And using result of analysis, it helps users for making decisions.

The Framework of Stream Data Processing System for Realtime Health Care Service (실시간 헬스케어 서비스를 위한 스트림 데이터 시스템 프레임워크의 설계)

  • Wu, Zejun;Lee, Yeon;Bae, Hae-Young
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2011.06a
    • /
    • pp.21-22
    • /
    • 2011
  • The growth of using smartphone and tablet pc has enabled variety kinds of realtime applications. In these applications, the data which we called data stream is multidimensional, continuous, rapid, and time-varying. However the traditional Database Management System (DBMS) suffers from processing the real time and complex application, in this paper we proposed the framework for CCR Data Stream Server's design and implementation that compiled with Data Stream Database Management System (DSMS) and DBMS in EMR system. The system enables users not only to query stored CCR information from DBMS, but also to execute continues query for the real-time CCR Data Stream.

  • PDF

Simulation Study on the Stream Server for Deciding the Priority for Using Resources (스트림 서버에서 자원 사용 우선순위 결정을 위한 시뮬레이션 연구)

  • 박진원
    • Journal of the Korea Society for Simulation
    • /
    • v.12 no.4
    • /
    • pp.95-102
    • /
    • 2003
  • Stream servers are for supplying multimedia stream data to users through the internet such as movies and music without discontinuation. A typical stream server is designed roughly by considering the characteristics of stream services and by employing processors, memory, PCI bus, Ethernet, TOE and disks. This study focuses on deciding the priority for using resources such as PCI bus, buffer memory and TOE buffer, which have limited capacities in a typical stream server. When the priorities for using limited resources are not given properly, the stream servers may not even function as originally designed. The simulation study shows that the top priority for using PCI bus for normal streaming services should be given to the operation that sends data from buffer memory to TOE buffer. Giving priority for using PCI bus to other operation such as sending data from disks to memory results in a deadlock phenomenon.

  • PDF

Causality join query processing for data stream by spatio-temporal sliding window (시공간 슬라이딩윈도우기법을 이용한 데이터스트림의 인과관계 결합질의처리방법)

  • Kwon, O-Je;Li, Ki-Joune
    • Spatial Information Research
    • /
    • v.16 no.2
    • /
    • pp.219-236
    • /
    • 2008
  • Data stream collected from sensors contain a large amount of useful information including causality relationships. The causality join query for data stream is to retrieve a set of pairs (cause, effect) from streams of data. A part of causality pairs may however be lost from the query result, due to the delay from sensors to a data stream management system, and the limited size of sliding windows. In this paper, we first investigate spatial, temporal, and spatio-temporal aspects of the causality join query for data stream. Second, we propose several strategies for sliding window management based on these observations. The accuracy of the proposed strategies is studied by intensive experiments, and the result shows that we improve the accuracy of causality join query in data stream from simple FIFO strategy.

  • PDF

Development of the Performance Benchmark Tool for Data Stream Management Systems Combined with DBMS (DBMS와 결합된 데이터스트림관리시스템을 위한 성능 평가 도구 개발)

  • Kim, Gyoung-Bae
    • Journal of the Korea Society of Computer and Information
    • /
    • v.15 no.8
    • /
    • pp.1-11
    • /
    • 2010
  • Many applications of DSMS(Data Stream Management System) require not only to process real-time stream data efficiently but also to provide high quality services such as data mining and data warehouse combining with DBMS(Database Management System) to users. In this paper we execute the performance benchmark of the combined system of DSMS and DBMS that is developed for high quality services. We use the stream data of network monitoring application system and combine the traditional representative DSMSs and DBMSs in a single system for the performance testing. We develop the total performance benchmark tool implementing JAVA language for the our testing. For our performance testing, we combine DSMS such as STREAM and Coral8 and DBMS such MySQL and Oracle10g respectively.

Simulation Study on the Stream Server for Deciding the Priority for Using Resources (스트림 서버에서 자원 사용 우선순위 결정을 위한 시뮬레이션 연구)

  • 박진원
    • Proceedings of the Korea Society for Simulation Conference
    • /
    • 2003.11a
    • /
    • pp.67-74
    • /
    • 2003
  • Stream servers are for supplying multimedia stream data to users through the internet such as movies and musics without discontinuation. A typical stream server is designed roughly by considering the characteristics of stream services and by employing processors, memory, PCI bus, Ethernet, TOE and disks. This study focuses on deciding the priority for using resources such as PCI bus, buffer memory and TOE buffer, which have limited capacities in a typical stream server. The simulation study shows that the top priority for using PCI bus for normal streaming services should be given to the operation that sends data from buffer memory to TOE buffer Giving priority for using PCI bus to other operation such as sending data from disks to memory results in deadlock Phenomenon.

  • PDF

Mining Association Rules in Multidimensional Stream Data (다차원 스트림 데이터의 연관 규칙 탐사 기법)

  • Kim, Dae-In;Park, Joon;Kim, Hong-Ki;Hwang, Bu-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.13D no.6 s.109
    • /
    • pp.765-774
    • /
    • 2006
  • An association rule discovery, a technique to analyze the stored data in databases to discover potential information, has been a popular topic in stream data system. Most of the previous researches are concerned to single stream data. However, this approach may ignore in mining to multidimensional stream data. In this paper, we study the techniques discovering the association rules to multidimensional stream data. And we propose a AR-MS method reflecting the characteristics of stream data since make the summarization information by one data scan and discovering the association rules for significant rare data that appear infrequently in the database but are highly associated with specific event. Also, AR-MS method can discover the maximal frequent item of multidimensional stream data by using the summarization information. Through analysis and experiments, we show that AR-MS method is superior to other previous methods.

Concept Drift Based on CNN Probability Vector in Data Stream Environment

  • Kim, Tae Yeun;Bae, Sang Hyun
    • Journal of Integrative Natural Science
    • /
    • v.13 no.4
    • /
    • pp.147-151
    • /
    • 2020
  • In this paper, we propose a method to detect concept drift by applying Convolutional Neural Network (CNN) in a data stream environment. Since the conventional method compares only the final output value of the CNN and detects it as a concept drift if there is a difference, there is a problem in that the actual input value of the data stream reacts sensitively even if there is no significant difference and is incorrectly detected as a concept drift. Therefore, in this paper, in order to reduce such errors, not only the output value of CNN but also the probability vector are used. First, the data entered into the data stream is patterned to learn from the neural network model, and the difference between the output value and probability vector of the current data and the historical data of these learned neural network models is compared to detect the concept drift. The proposed method confirmed that only CNN output values could be used to reduce detection errors compared to how concept drift were detected.