• Title/Summary/Keyword: Data Stream Process

Search Result 305, Processing Time 0.026 seconds

Dynamic Data Cubes Over Data Streams (데이타 스트림에서 동적 데이타 큐브)

  • Seo, Dae-Hong;Yang, Woo-Sock;Lee, Won-Suk
    • Journal of KIISE:Databases
    • /
    • v.35 no.4
    • /
    • pp.319-332
    • /
    • 2008
  • Data cube, which is multi-dimensional data model, have been successfully applied in many cases of multi-dimensional data analysis, and is still being researched to be applied in data stream analysis. Data stream is being generated in real-time, incessant, immense, and volatile manner. The distribution characteristics of data arc changing rapidly due to those characteristics, so the primary rule of handling data stream is to check once and dispose it. For those characteristics, users are more interested in high support attribute values observed rather than the entire attribute values over data streams. This paper propose dynamic data cube for applying data cube to data stream environment. Dynamic data cube specify user's interested area by the support ratio of attribute value, and dynamically manage the attribute values by grouping each other. By doing this it reduce the memory usage and process time. And it can efficiently shows or emphasize user's interested area by increasing the granularity for attributes that have higher support. We perform experiments to verify how efficiently dynamic data cube works in limited memory usage.

A method of event data stream processing for ALE Middleware (ALE 미들웨어를 위한 이벤트 데이터 처리 방법)

  • Noh, Young-Sik;Lee, Dong-Cheol;Byun, Yung-Cheol
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.12 no.9
    • /
    • pp.1554-1563
    • /
    • 2008
  • As the interests on RFID technologies increase, a lot of research activities on RFID middleware systems to handle the data acquired by RFID readers are going on actively. Meanwhile, even though various kinds of RFID middleware methodologies and related techniques have been proposed, the common data type which is dealt with in those systems is an EPC code, mainly. Also, there are few researches of the implementation of collecting the stream data queued from RFID readers endlessly and without blocking, classifying the data into some groups according to usage, and sending the resulting data to specific applications. In this paper, we propose the method of data handling in RFID middleware to efficiently process an EPC event stream data using detail filtering, checking of data modification, creation of data set to transfer, data grouping, and various kinds of RFID data format transform. Our method is based on a de facto international standard interface defined in the ALE middleware specification by EPCglobal, and application and service users can directly set various kinds of conditions to handle the stream data.

Efficient Processing of Multidimensional Vessel USN Stream Data using Clustering Hash Table (클러스터링 해쉬 테이블을 이용한 다차원 선박 USN 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Oh, Il-Whan;Lee, Seong-Ro
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.47 no.6
    • /
    • pp.137-145
    • /
    • 2010
  • Digital vessel have to accurate and efficient mange the digital data from various sensors in the digital vessel. But, In sensor network, it is difficult to transmit and analyze the entire stream data depending on limited networks, power and processor. Therefore it is suitable to use alternative stream data processing after classifying the continuous stream data. In this paper, We propose efficient processing method that arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and pre-clustering using multiple Support Vector Machine(SVM) algorithm and manage hash table to summarized information. Processing performance improve as store and search and memory using hash table and usage reduced so maintain hash table in memory. We obtained to efficient result that accuracy rate and processing performance of proposal method using 35,912 data sets.

A fast watermark embedding method for MPEG-2 bit stream (MPEG-2 비트 스트림에 대한 고속 워터마크 삽입방법)

  • 김성일;서정일;김구영;원치선
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 1997.11a
    • /
    • pp.151-154
    • /
    • 1997
  • In this paper, we propose a new watermarking algorithm for copyright protection of video data. The proposed algorithm inserts a watermark directly on the MPEG-2 bitstream. Since more and more video data are stored and transmitted in a compressed form, it is desirable to insert a watermark on the compressed bit stream to avoid the expensive full-decoding and re-encoding process. Embedding a watermark in the compressed domain, we can also avoid the effect of the compression error which may erase the watermark.

  • PDF

Assessment of Degree of Naturalness of Vegetation on the Riverine Wetland (하천습지의 식생학적 자연도 평가)

  • Chun, Seung-Hoon
    • Journal of Environmental Impact Assessment
    • /
    • v.20 no.1
    • /
    • pp.1-11
    • /
    • 2011
  • This study was carried out to suggest the baseline data necessary for vegetation restoration at riverine wetland within stream corridor. We used the prevalence index for wetland assessment by applying the method of weighted averages with index values based on five hydrophyte indicator status as defined by estimated probability occurred in wetland. We selected near nature and urbanized reach of Gap and Yanghwa streams as experimental site. Although two sites have some different disturbance and characteristics of watershed, they showed that similarity of vegetation community including three dominant species - Salix koreensis, Phragmites communis, Miscanthus sacchariflorus - was very high. But in case of Yanghwa stream, various kinds of emergent plants along wetted condition were distinctly occurred, resulted from difference of hydrological regime and substrate, etc. Degree of naturalness of vegetation at the sampled areas indicated that near nature area of Gap stream and all area of Yanghwa stream were fitted as riverine wetland, while urbanized area of Gap stream has changed into upland condition. In conclusion assessment system using prevalence index would be considered an effective method for evaluating of natural states of riverine wetland, but further integrated consideration of physical, hydrological, and biological factors of stream process, and also with considering the difference between those qualitative data of vegetation community.

Development of a Hybrid Watershed Model STREAM: Test Application of the Model (복합형 유역모델 STREAM의 개발(II): 모델의 시험 적용)

  • Cho, Hong-Lae;Jeong, Euisang;Koo, Bhon Kyoung
    • Journal of Korean Society on Water Environment
    • /
    • v.31 no.5
    • /
    • pp.507-522
    • /
    • 2015
  • In this study, some of the model verification results of STREAM (Spatio-Temporal River-basin Ecohydrology Analysis Model), a newly-developed hybrid watershed model, are presented for the runoff processes of nonpoint source pollution. For verification study of STREAM, the model was applied to a test watershed and a sensitivity analysis was also carried out for selected parameters. STREAM was applied to the Mankyung River Watershed to review the applicability of the model in the course of model calibration and validation against the stream flow discharge, suspended sediment discharge and some water quality items (TOC, TN, TP) measured at the watershed outlet. The model setup, simulation and data I/O modules worked as designed and both of the calibration and validation results showed good agreement between the simulated and the measured data sets: NSE over 0.7 and $R^2$ greater than 0.8. The simulation results also include the spatial distribution of runoff processes and watershed mass balance at the watershed scale. Additionally, the irrigation process of the model was examined in detail at reservoirs and paddy fields.

Video Stream Smoothing Using Multistreams (멀티스트림을 이용한 비디오 스트림의 평활화)

  • 강경원;문광석
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.3 no.1
    • /
    • pp.21-26
    • /
    • 2002
  • Video stream invoke a variety of traffic with the structure of compression algorithm and image complexity. Thus, it is difficult to allocate the resource on the both sides of sender and receiver, and playout on the Internet such as a packet switched network. Thus, in this paper we proposed video stream smoothing using multistream for the effective transmission of video stream. This method specifies the type of LDU(logical data unit) according to the type of original stream, and then makes a large number of streams as a fixed size, and transfers them. So, the proposed method can reduce the buffering time which occurs during the process of the smoothing and prefetch be robust to the jitter on network, as well. Consequently, it has the effective transmission characteristics of fully utilizing the clients bandwidth.

  • PDF

Development of a Spatial DSMS for Efficient Real-Time Processing of Spatial Sensor Data (공간 센서 데이타의 효율적인 실시간 처리를 위한공간 DSMS의 개발)

  • Kang, Hong-Koo;Park, Chi-Min;Hong, Dong-Suk;Han, Ki-Joon
    • Journal of Korea Spatial Information System Society
    • /
    • v.9 no.1
    • /
    • pp.45-57
    • /
    • 2007
  • Recently, the development of sensor devices has accelerated researches on advanced technologies like Wireless Sensor Networks. Moreover, spatial sensors using GPS lead to the era of the Ubiquitous Computing Environment which generally uses spatial information and non-spatial information together. In this new era, a real-time processing system for spatial sensor data is essential. In this reason, new data processing systems called DSMS(Data Stream Management System) are being developed by many researchers. However, since most of them do not support geometry types and spatial functions to process spatial sensor data, they are not suitable for the Ubiquitous Computing Environment. For these reasons, in this paper, we designed and implemented a spatial DSMS by extending STREAM which stands for STanford stREam datA Manager, to solve these problems. We added geometry types and spatial functions to STREAM in order to process spatial sensor data efficiently. In addition, we implemented a Spatial Object Manager to manage shared spatial objects within the system. Especially, we implemented the Simple Features Specification for SQL of OGC for interoperability and applied algorithms in GEOS to our system.

  • PDF

An Efficient Method for Mining Frequent Patterns based on Weighted Support over Data Streams (데이터 스트림에서 가중치 지지도 기반 빈발 패턴 추출 방법)

  • Kim, Young-Hee;Kim, Won-Young;Kim, Ung-Mo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.10 no.8
    • /
    • pp.1998-2004
    • /
    • 2009
  • Recently, due to technical developments of various storage devices and networks, the amount of data increases rapidly. The large volume of data streams poses unique space and time constraints on the data mining process. The continuous characteristic of streaming data necessitates the use of algorithms that require only one scan over the stream for knowledge discovery. Most of the researches based on the support are concerned with the frequent itemsets, but ignore the infrequent itemsets even if it is crucial. In this paper, we propose an efficient method WSFI-Mine(Weighted Support Frequent Itemsets Mine) to mine all frequent itemsets by one scan from the data stream. This method can discover the closed frequent itemsets using DCT(Data Stream Closed Pattern Tree). We compare the performance of our algorithm with DSM-FI and THUI-Mine, under different minimum supports. As results show that WSFI-Mine not only run significant faster, but also consume less memory.

Identification of Urban Stream Sandbar Change After Concentrated Storm during Summer (집중호우 후 도시 자연형하천의 사주변화 파악)

  • Kim, Jae-Cheol;Lee, Sang-Hwa;Shin, Dong-Hoon;Lee, Kyoo-Seock
    • Journal of the Korean Society of Environmental Restoration Technology
    • /
    • v.9 no.6
    • /
    • pp.162-167
    • /
    • 2006
  • The urban stream includes the channel and sandbars. The sandbar plays a key role in the riparian ecosystem. For birds and insects the sandbar offers a small strip of habitat and fish and other fauna feed in the boundary of sandbar where eddies occur. So, it is important habitat and source for the flow of energy, matter and organisms through the landscape and act as ecotone between the terrestrial and stream corridors. However, the sandbar changes continuously by the natural process. Thus, it is necessary to measure the shape and area of the sandbar accurately for the efficient urban stream management for the amenity of urban residents and stream protection. The study site is Yangjae Stream where the first natural-style urban stream restoration projected was impelemented by the support of Ministry of Education in Korea. The measurement was taken by the beacon Differential Global Positioning System (DGPS) and the data were stored and analyzed using ArcView Geographic Information System (GIS) program. Therefore, the purpose of this study is to measure the change of sandbars in the urban stream after concentrated stormwater during summer.