• Title/Summary/Keyword: Data stream

Search Result 2,528, Processing Time 0.03 seconds

A Method of Frequent Structure Detection Based on Active Sliding Window (능동적 슬라이딩 윈도우 기반 빈발구조 탐색 기법)

  • Hwang, Jeong-Hee
    • Journal of Digital Contents Society
    • /
    • v.13 no.1
    • /
    • pp.21-29
    • /
    • 2012
  • In ubiquitous computing environment, rising large scale data exchange through sensor network with sharply growing the internet, the processing of the continuous stream data is required. Therefore there are some mining researches related to the extracting of frequent structures and the efficient query processing of XML stream data. In this paper, we propose a mining method to extract frequent structures of XML stream data in recent window based on the active window sliding using trigger rule. The proposed method is a basic research to control the stream data flow for data mining and continuous query by trigger rules.

Performance Optimization Considering I/O Data Coherency in Stream Processing (Stream Processing에서 I/O데이터 일관성을 고려한 성능 최적화)

  • Na, Hana;Yi, Joonwhan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.53 no.8
    • /
    • pp.59-65
    • /
    • 2016
  • Performance optimization of applications with massive stream data processing has been performed by considering I/O data coherency problem where a memory is shared between processors and hardware accelerators. A formula for performance analyses is derived based on profiling results of system-level simulations. Our experimental results show that overall performance was improved by 1.40 times on average for various image sizes. Also, further optimization has been performed based on the parameters appeared in the derived formula. The final performance gain was 3.88 times comparing to the original design and we can find that the performance of the design with cacheable shared memory is not always.

A Real-Time Data Mining for Stream Data Sets (연속발생 데이터를 위한 실시간 데이터 마이닝 기법)

  • Kim Jinhwa;Min Jin Young
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.29 no.4
    • /
    • pp.41-60
    • /
    • 2004
  • A stream data is a data set that is accumulated to the data storage from a data source over time continuously. The size of this data set, in many cases. becomes increasingly large over time. To mine information from this massive data. it takes much resource such as storage, memory and time. These unique characteristics of the stream data make it difficult and expensive to use this large size data accumulated over time. Otherwise. if we use only recent or part of a whole data to mine information or pattern. there can be loss of information. which may be useful. To avoid this problem. we suggest a method that efficiently accumulates information. in the form of rule sets. over time. It takes much smaller storage compared to traditional mining methods. These accumulated rule sets are used as prediction models in the future. Based on theories of ensemble approaches. combination of many prediction models. in the form of systematically merged rule sets in this study. is better than one prediction model in performance. This study uses a customer data set that predicts buying power of customers based on their information. This study tests the performance of the suggested method with the data set alone with general prediction methods and compares performances of them.

CONTINUOUS QUERY PROCESSING IN A DATA STREAM ENVIRONMENT

  • Lee, Dong-Gyu;Lee, Bong-Jae;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2007.10a
    • /
    • pp.3-5
    • /
    • 2007
  • Many continuous queries are important to be process efficiently in a data stream environment. It is applied a query index technique that takes linear performance irrespective of the number and width of intervals for processing many continuous queries. Previous researches are not able to support the dynamic insertion and deletion to arrange intervals for constructing an index previously. It shows that the insertion and search performance is slowed by the number and width of interval inserted. Many intervals have to be inserted and searched linearly in a data stream environment. Therefore, we propose Hashed Multiple Lists in order to process continuous queries linearly. Proposed technique shows fast linear search performance. It can be utilized the systems applying a sensor network, and preprocessing technique of spatiotemporal data mining.

  • PDF

Generation and Interpretation of data stream for position data of objects synchronized with video (비디오와 동기화된 물체의 위치정보 표현 data stream 생성 및 해석기 구현)

  • Na, Hee-Joo;Kim, Jung-Hwan;Jung, Moon-Ryul
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2005.11a
    • /
    • pp.249-254
    • /
    • 2005
  • 본 논문은 디지털 방송 프로그램 진행 중 비디오의 특정 시점에 동기화된 특정 객체의 위치정보를 표현하는 data stream을 생성하고, 그 시점에 해당 위치 정보를 해석하는 해석기에 관한 것이다. 현재의 상용 스트림 생성기는 디지털 방송 표준에서 권고하는 스트림 이벤트의 발생 시각과 셋톱박스에서 디코딩 시에 사용할 참조값을 적절하게 생성하지 못하고 있다. 또한, 셋톱박스에서 동작하는 애플리케이션(Xlet) 역시 STC(System Time Clock), PCR(Program Clock Reference), NPT(Normal Play Time) 등의 시간값을 적절하게 읽어내지 못하고 있다. 더욱이, 현재의 디지털 방송 표준에서는 영상 내 특정 객체를 위해 정보를 제공하는 데에는 한계가 있다. 따라서, 본 논문에서는 다양한 연동형 디지털 방송 프로그램 제작을 위해 비디오의 특정 시점에 동기화된 객체의 위치정보를 표현하는 data stream을 생성하는 방법과, 동기화된 데이터를 처리하는 애플리케이션에 대해서 설명한다.

  • PDF

An Effective Stream Data Management System for the Incomplete Stream Data on Sensor Network (센서 네트워크에서의 불완전 스트림데이터를 위한 효율적인 스트림 데이터 관리 시스템)

  • Park, Eun-Ji;Byeon, Jeong-Woo;Choi, Da-Som;Kim, Jin-Han;Oh, Ryum-Duck
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2014.01a
    • /
    • pp.125-126
    • /
    • 2014
  • 센서 스트림 데이터는 센서 네트워크를 통해 수집되는 데이터로 실시간 처리를 요구하며, 연속적으로 끊임없이 발생하는 스트림 데이터이다. 이러한 스트림 데이터는 양이 방대하여 이를 저장하기가 매우 어려우며, 동시에 데이터를 검색하는 데에는 많은 시간이 소요된다. 본 논문에서는 센서 네트워크에서의 효율적인 스트림 데이터 처리 시스템을 제안한다. 이 시스템은 캐시테이블을 사용함으로써 데이터베이스에 최소화된 접근으로 데이터 스트림 관리 시스템의 성능을 개선하였다. 그리고 센서 네트워크에서 읽어 들여온 불완전 데이터를 효율적으로 정제하고 상위 단계로 전송한다.

  • PDF

Development of Stream Width and Bed-slope Estimation Equations for Preparing Data for Distributed Storm Runoff Model (분포형 강우-유출모형의 하도자료 구축을 위한 하폭 및 하상경사 산정공식 개발)

  • Jung, In-Kyun;Park, Jong-Yoon;Joh, Hyung-Kyung;Lee, Ji-Wan;Kim, Seong-Joon
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.52 no.4
    • /
    • pp.1-10
    • /
    • 2010
  • In this study, two estimation equations for preparing stream data for distributed storm runoff model were developed by analyzing the nonlinear relation between upstream flow-length and stream width, and between upstream flow-length and stream bed-slope. The equations for stream cell were tested in Chungjudam watershed (6,661 $km^2$) using KIMSTORM. Six storm events occurring between 2003 and 2008 were selected for the model calibration and verification before the test of equations. The average values of the Nash-Sutcliffe model efficiency (ME), the volume conservation index (VCI), the relative error of peak runoff rate (EQp), and the difference of time to peak runoff (DTp) were 0.929, 1.035, 0.037, and -0.406 hr for the calibrated four storm events and 0.956, 0.939, 0.055, and 0.729 hr for the two verified storm events respectively. The estimation equations were tested to the storm events, and compared the flood hydrograph. The test result showed that the estimation equation of stream width reduced the peak runoff and delaying the time to peak runoff, and the estimation equation of stream bed-slope showed the opposite results.

A Study on Transport Stream Analysis and Parsing Ability Enhancement in Digital Broadcasting and Service (디지털 방송 서비스에서 트랜스포트 스트림 분석 및 파싱 능력 향상에 관한 연구)

  • Kim, Jang-Won
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.10 no.6
    • /
    • pp.552-557
    • /
    • 2017
  • Wire, wireless digital broadcasting has sharply expanded with the birth of high definition TV since 2010, the use of duplex contents as well as simplex contents has rapidly increased. Currently, our satellite communications system adopted DVB by European digital broadcasting standardization organization as a standard of domestic data broadcasting, the method how to use selective contents has been studied variously according to the development of IPTV. Digital broadcasting utilizes the method using Transport Stream Packet(TSP) by the way of multiplexing of information in order to send multimedia information such as video, audio and data of MPEG-2, this streams include detail information on TV guide and program as well as video and audio information. In order to understand these data broadcasting system, this study realized TS analyzer that divides transport stream (TS) by packet in Linux environment, analyzes and prints by function, it can help the understanding of TS, the enhancement of stream parsing ability.

Design and Implementation of the Spatio-Temporal DSMS for Moving Object Data Streams (이동체 데이타 스트림을 위한 시공간 DSMS의 설계 및 구현)

  • Lee, Ki-Young;Kim, Joung-Joon
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.8 no.5
    • /
    • pp.159-166
    • /
    • 2008
  • Recently, according to the rapid development of location positioning technology and wireless communications technology and increasement of usage of moving object data, many researches and developments on the real-time locating systems which provides real time service of moving object data stream are under proceeding. However, MO (Moving Object) DBMS used based system in the in these systems is the inefficient management of moving object data streams, and the existing DSMS (Data Stream Management System) has problems that spatio-temporal data are not handled efficiently. Therefore, in this thesis, we designed and implemented spatio-temporal DSMS for efficient real-time management of moving object data stream. This thesis implemented spatio-temporal DSMS based STREAM (STanford stREam dAta Manager) of Stanford University is supporting real-time management of moving object data stream and spatio-temproal query processing and filtering for reduce the input loading. Specifically, spatio-temporal operators of the spatio-temporal DSMS support standard interface of SQL form which extended "Simple Feature Specification for SQL" standard specifications presented by OGC for compatibility. Finally, implemented spatio-temporal DSMS in this thesis, proved the effectiveness of the system that as applied real-time monitoring areas that require real-time locating of object data stream DSMS.

  • PDF

Design of Sensor Middleware Architecture on Multi Level Spatial DBMS with Snapshot (스냅샷을 가지는 다중 레벨 공간 DBMS를 기반으로 하는 센서 미들웨어 구조 설계)

  • Oh, Eun-Seog;Kim, Ho-Seok;Kim, Jae-Hong;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.8 no.1 s.16
    • /
    • pp.1-16
    • /
    • 2006
  • Recently, human based computing environment for supporting users to concentrate only user task without sensing other changes from users is being progressively researched and developed. But middleware deletes steream data processed for reducing process load of massive information from RFID sensor in this computing. So, this kind of middleware have problems when user demands probability or statistics needed for data warehousing or data mining and when user demands very important stream data repeatedly but already discarded in the middleware every former time. In this paper, we designs Sensor Middleware Architecture on Multi Level Spatial DBMS with Snapshot and manage repeatedly required stream datas to solve reusing problems of historical stream data in current middleware. This system uses disk databse that manages historical stream datas filtered in middleware for requiring services using historical stream information as data mining or data warehousing from user, and uses memory database that mamages highly reuseable data as a snapshot when stream data storaged in disk database has high reuse frequency from user. For the more, this system processes memory database management policy in a cycle to maintain high reusement and rapid service for users. Our paper system solves problems of repeated requirement of stream datas, or a policy decision service using historical stream data of current middleware. Also offers variant and rapid data services maintaining high data reusement of main memory snapshot datas.

  • PDF