• Title/Summary/Keyword: Stream Data

Search Result 2,514, Processing Time 0.026 seconds

Context Inference and Sensor Data Classification of Big Data Stream Environment (빅데이터 스트림 환경에서의 센서 데이터 분류와 상황추론)

  • Ryu, Chang-Kun
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.9 no.10
    • /
    • pp.1079-1085
    • /
    • 2014
  • The analysis of the variable continuous big data stram should reach the destination context awareness. This study presented a novel way of context inference of the variable data stream from sensor motes. For assessment of the sensor data, we calculated the difference of each measured value at the time window and determined the belief value of each focal element. It was beneficial that calculate and assessment of factor of situation for context inference with the Dempster-Shfer evidence theory.

Mining Frequent Itemsets with Normalized Weight in Continuous Data Streams

  • Kim, Young-Hee;Kim, Won-Young;Kim, Ung-Mo
    • Journal of Information Processing Systems
    • /
    • v.6 no.1
    • /
    • pp.79-90
    • /
    • 2010
  • A data stream is a massive unbounded sequence of data elements continuously generated at a rapid rate. The continuous characteristic of streaming data necessitates the use of algorithms that require only one scan over the stream for knowledge discovery. Data mining over data streams should support the flexible trade-off between processing time and mining accuracy. In many application areas, mining frequent itemsets has been suggested to find important frequent itemsets by considering the weight of itemsets. In this paper, we present an efficient algorithm WSFI (Weighted Support Frequent Itemsets)-Mine with normalized weight over data streams. Moreover, we propose a novel tree structure, called the Weighted Support FP-Tree (WSFP-Tree), that stores compressed crucial information about frequent itemsets. Empirical results show that our algorithm outperforms comparative algorithms under the windowed streaming model.

Generalization of the Stream Network by the Geographic Hierarchy of Landform Data (지형자료의 계층화를 이용한 하계망 일반화)

  • Kim Nam-Shin
    • Journal of the Korean Geographical Society
    • /
    • v.40 no.4 s.109
    • /
    • pp.441-453
    • /
    • 2005
  • This study aims to generalize the stream network developing algorithm of the geographic hierarchy Stream networks with hierarchy system should be spatially hierarchized in linear features. The generalization procedure of the stream networks are composed of the hierarchy of stream, selection and elimination, and algorithm. Working of stream networks is composed by the decision of direction on stream networks, ranking of stroke segments, and ordering by the strahler method, using geographic data query for controlling selection and elimination of the linear feature by scale. Improved Simoo algorithm was effective in enhancement and decreasing curvature of linear features. Resultantly, it is expected to improve generalization of features with various spatial hierarchy.

Efficient Processing of Multidimensional Sensor stream Data in Digital Marine Vessel (디지털 선박 내 다차원 센서 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Park, Kyung-Woo;Lee, Jin-Seok;Lee, Keong-Hyo;Jung, Min-A;Lee, Sung-Ro
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.5B
    • /
    • pp.794-800
    • /
    • 2010
  • It is necessary to accurate and efficient management for measured digital data from various sensors in digital marine vessel. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. In this paper, We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose that we arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using SVM algorithm. We automatically delete that it isn't necessary to the data from the database and we used to ship diagnosis system for available data. As a result, we obtained to efficient result about 18.3% reduction rate of database using 35,912 data sets.

Adaptive Upstream Backup Scheme based on Throughput Rate in Distributed Spatial Data Stream System (분산 공간 데이터 스트림 시스템에서 연산 처리율 기반의 적응적 업스트림 백업 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.10
    • /
    • pp.5156-5161
    • /
    • 2013
  • In distributed spatial data stream processing, processed tuples of downstream nodes are replicated to the upstream node in order to increase the utilization of distributed nodes and to recover the whole system for the case of system failure. However, while the data input rate increases and multiple downstream nodes share the operation result of the upstream node, the data which stores to output queues as a backup can be lost since the deletion operation delay may be occurred by the delay of the tuple processing of upstream node. In this paper, the adaptive upstream backup scheme based on operation throughput in distributed spatial data stream system is proposed. This method can cut down the average load rate of nodes by efficient spatial operation migration as it processes spatial temporal data stream, and it can minimize the data loss by fluid change of backup mode. The experiments show the proposed approach can prevent data loss and can decrease, on average, 20% of CPU utilization by node monitoring.

An Efficient Method for Mining Frequent Patterns based on Weighted Support over Data Streams (데이터 스트림에서 가중치 지지도 기반 빈발 패턴 추출 방법)

  • Kim, Young-Hee;Kim, Won-Young;Kim, Ung-Mo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.10 no.8
    • /
    • pp.1998-2004
    • /
    • 2009
  • Recently, due to technical developments of various storage devices and networks, the amount of data increases rapidly. The large volume of data streams poses unique space and time constraints on the data mining process. The continuous characteristic of streaming data necessitates the use of algorithms that require only one scan over the stream for knowledge discovery. Most of the researches based on the support are concerned with the frequent itemsets, but ignore the infrequent itemsets even if it is crucial. In this paper, we propose an efficient method WSFI-Mine(Weighted Support Frequent Itemsets Mine) to mine all frequent itemsets by one scan from the data stream. This method can discover the closed frequent itemsets using DCT(Data Stream Closed Pattern Tree). We compare the performance of our algorithm with DSM-FI and THUI-Mine, under different minimum supports. As results show that WSFI-Mine not only run significant faster, but also consume less memory.

A Push-Caching and a Transmission Scheme of Continuous Media for NOD Service on the Internet (인테넷상에서 NOD 서비스를 위한 연속미디어 전송 및 푸쉬-캐싱 기법)

  • Park, Seong-Ho;Im, Eun-Ji;Choe, Tae-Uk;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.6
    • /
    • pp.1766-1777
    • /
    • 2000
  • In multimedia new service on the internet, there are problems such as server overload, network congestion and initial latency. To overcome these problems, we propose a proxy push-caching scheme that stores a portion of continuous media stream or entire stream, and a transmission scheme of NOD continuous media, RTP-RR and RTP-nR to exploit push-caching scheme. With the proposed push-caching scheme, NOD server pushes fixed portion of stream to a proxy when new data is generated, and the cached size of each stream changes dynamically according to the caching utility value of each stream. As a result, the initial latency of client side could be reduced and the amount of data transmitted fro ma proxy server to client could be increased. Moreover, we estimate a caching utility value of each stream using correlation between disk space occupied by the stream and the amount of data stream requested by client. And we applied the caching utility value ot replacement policies. The performance of the proxy push-caching and continuous media transmission schemes proposed were compared with other schemes using simulations. In the simulation, these schemes show better results than other schemes in terms of BHR (Byte Hit Rate), initial latency, the number of replacement and packet loss rate.

  • PDF

H*-tree/H*-cubing-cubing: Improved Data Cube Structure and Cubing Method for OLAP on Data Stream (H*-tree/H*-cubing: 데이터 스트림의 OLAP를 위한 향상된 데이터 큐브 구조 및 큐빙 기법)

  • Chen, Xiangrui;Li, Yan;Lee, Dong-Wook;Kim, Gyoung-Bae;Bae, Hae-Young
    • The KIPS Transactions:PartD
    • /
    • v.16D no.4
    • /
    • pp.475-486
    • /
    • 2009
  • Data cube plays an important role in multi-dimensional, multi-level data analysis. Meeting on-line analysis requirements of data stream, several cube structures have been proposed for OLAP on data stream, such as stream cube, flowcube, S-cube. Since it is costly to construct data cube and execute ad-hoc OLAP queries, more research works should be done considering efficient data structure, query method and algorithms. Stream cube uses H-cubing to compute selected cuboids and store the computed cells in an H-tree, which form the cuboids along popular-path. However, the H-tree layoutis disorderly and H-cubing method relies too much on popular path.In this paper, first, we propose $H^*$-tree, an improved data structure, which makes the retrieval operation in tree structure more efficient. Second, we propose an improved cubing method, $H^*$-cubing, with respect to computing the cuboids that cannot be retrieved along popular-path when an ad-hoc OLAP query is executed. $H^*$-tree construction and $H^*$-cubing algorithms are given. Performance study turns out that during the construction step, $H^*$-tree outperforms H-tree with a more desirable trade-off between time and memory usage, and $H^*$-cubing is better adapted to ad-hoc OLAP querieswith respect to the factors such as time and memory space.

Fast Speaker Adaptation Using Sub-Stream Based Eigenvoice (Sub-Stream 기반의 Eigenvoice를 이용한 고속 화자적응)

  • Song, Hwa-Jeon;Lee, Jong-Seok;Kim, Hyung-Soon
    • MALSORI
    • /
    • v.55
    • /
    • pp.93-102
    • /
    • 2005
  • In this paper, sub-stream based eigenvoice method is proposed to overcome the weak points of conventional eigenvoice and dimensional eigenvoice. In the proposed method, sub-streams are automatically constructed by the statistical clustering analysis that uses the correlation information between dimensions. To obtain the reliable distance matrix from covariance matrix for dividing into optimal sub-streams, MAP adaptation technique is employed to the covariance matrix of training data and the sample covariance of adaptation data. According to our experiments, the proposed method shows $41\%$ error rate reduction when the number of adaptation data is 50.

  • PDF

Practical Patching for Efficient Bandwidth Sharing in VOD Systems

  • Ha Soak-Jeong
    • Journal of Korea Multimedia Society
    • /
    • v.8 no.12
    • /
    • pp.1597-1604
    • /
    • 2005
  • Recursive Patching is an efficient multicast technique for large-scale video on demand systems and recursively shares existing video streams with asynchronous clients. When Recursive Patching initiates a transition stream, it always makes a transition stream have additional data for the worst future request. In order to share a VOD server's limited network bandwidth efficiently, this paper proposes Practical Patching that removes the unnecessary data included in the transition stream. The proposed Practical Patching dynamically expands ongoing transition streams when a new request actually arrives at the server. As a result, the transition streams never have unnecessary data. Simulation result confirmed that the proposed technique is better than Recursive Patching in terms of service latency and defection rate.

  • PDF