• Title/Summary/Keyword: data streams

Search Result 821, Processing Time 0.025 seconds

Adaptive Buffer Control over Disordered Streams (비순서화된 스트림 처리를 위한 적응적 버퍼 제어 기법)

  • Kim, Hyeon-Gyu;Kim, Cheol-Gi;Lee, Chung-Ho;Kim, Myoung-Ho
    • Journal of KIISE:Databases
    • /
    • v.34 no.5
    • /
    • pp.379-388
    • /
    • 2007
  • Disordered streams may cause inaccurate or delayed results in window-based queries. Existing approaches usually leverage buffers to hand]e the streams. However, most of the approaches estimate the buffer size simply based on the maximum network delay in the streams, which tends to over-estimate the buffer size and result in high latency. In this paper, we propose a probabilistic approach to estimate the buffer size adaptively according to the fluctuated network delays. We first assume that intervals of tuple generations follow an exponential distribution and network delays have a normal distribution. Then, we derive an estimation function from the assumptions. The function takes a drop ratio as an input parameter, which denotes a percentage of tuple drops permissible during query execution. By describing the drop ratio in a query specification, users can control the quality of query results such as accuracy or latency according to application requirements. Our experimental results show that the proposed function has better adaptivity than the existing function based on the maximum network delay.

Similarity Search Algorithm Based on Hyper-Rectangular Representation of Video Data Sets (비디오 데이터 세트의 하이퍼 사각형 표현에 기초한 비디오 유사성 검색 알고리즘)

  • Lee, Seok-Lyong
    • The KIPS Transactions:PartD
    • /
    • v.11D no.4
    • /
    • pp.823-834
    • /
    • 2004
  • In this research, the similarity search algorithms are provided for large video data streams. A video stream that consists of a number of frames can be expressed by a sequence in the multidimensional data space, by representing each frame with a multidimensional vector By analyzing various characteristics of the sequence, it is partitioned into multiple video segments and clusters which are represented by hyper-rectangles. Using the hyper-rectangles of video segments and clusters, similarity functions between two video streams are defined, and two similarity search algorithms are proposed based on the similarity functions algorithms by hyper-rectangles and by representative frames. The former is an algorithm that guarantees the correctness while the latter focuses on the efficiency with a slight sacrifice of the correctness Experiments on different types of video streams and synthetically generated stream data show the strength of our proposed algorithms.

An Expanded Patching Technique using Four Types of Streams for True VoD Services

  • Ha, Sook-Jeong;Bae, Ihn-Han;Kim, Jin-Gyu;Park, Young-Ho;Oh, Sun-Jin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.3 no.5
    • /
    • pp.444-460
    • /
    • 2009
  • In this paper, we propose an expanded patching technique in order to reduce the server network bandwidth requirements to support true VoD services in VoD Systems. Double Patching, which is a typical multicast technique, ensures that a long patching stream delivers not only essential video data for the current client but also extra video data for future clients. Since the extra data may include useless data, it results in server network bandwidth wastage. In order to prevent a server from transmitting useless data, the proposed patching technique uses a new kind of stream called a linking stream. A linking stream is transmitted to clients that have received short patching streams, and it plays a linking role between a patching stream and a regular stream. The linking stream enables a server to avoid transmitting unnecessary data delivered by a long patching stream in Double Patching, so the server never wastes its network bandwidth. Mathematical analysis shows that the proposed technique requires less server network bandwidth to support true VoD services than Double Patching. Moreover, simulation results show that it has better average service latency and client defection rate compared with Double Patching.

Studies on the Stochastic Generation of Synthetic Streamflow Sequences(I) -On the Simulation Models of Streamflow- (하천유량의 추계학적 모의발생에 관한 연구(I) -하천유량의 Simulation 모델에 대하여-)

  • 이순탁
    • Water for future
    • /
    • v.7 no.1
    • /
    • pp.71-77
    • /
    • 1974
  • This paper reviews several different single site generation models for further development of a model for generating the Synthetic sequences of streamflow in the continuous streams like main streams in Korea. Initially the historical time series is looked using a time series technique, that is correlograms, to determine whether a lag one Markov model will satisfactorily represent the historical data. The single site models which were examined include an empirical model using the historical probability distribution of the random component, the linear autoregressive model(Markov model, or Thomas-Fiering model) using both logarithms of the data and Matala's log-normal transformation equations, and finally gamma distribution model.

  • PDF

A Multi-Query Optimizing Method for Data Stream Similar Queries on Sliding Window (슬라이딩 윈도에서의 데이터 스팀데이터 유사 질의 처리를 위한 다중질의 최적화 기법)

  • Liangbo Li;Yan Li;Song-Sun Shin;Dong-Wook Lee;Weon-Il Chung;Hae-Young Bae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2008.11a
    • /
    • pp.413-416
    • /
    • 2008
  • In the presence of multiple continuous queries, multi-query optimizing is a new challenge to process multiple stream data in real-time. So, in this paper, we proposed an approach to optimize multi-query of sliding window on network traffic data streams and do some comparisons to traditional queries without optimizing. We also detail some method of scheduling on different data streams, while different scheduling made different results. We test the results on variety of multi-query processing schedule, and proofed the proposed method is effectively optimized the data stream similar multi-queries.

An Efficient Join Algorithm for Data Streams with Overlapping Window (중첩 윈도우를 가진 데이터 스트링을 위한 효율적인 조인 알고리즘)

  • Kim, Hyeon-Gyu;Kang, Woo-Lam;Kim, Myoung-Ho
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.15 no.5
    • /
    • pp.365-369
    • /
    • 2009
  • Overlapping windows are generally used for queries to process continuous data streams. Nevertheless, existing approaches discussed join algorithms only for basic types of windows such as tumbling windows and tuple-driven windows. In this paper, we propose an efficient join algorithm for overlapping windows, which are considered as a more general type of windows. The proposed algorithm is based on an incremental window join. It focuses on producing join results continuously when the memory overflow frequently occurs. It consists of (1) a method to use both of the incremental and full joins selectively, (2) a victim selection algorithm to minimize latency of join processing and (3) an idle time professing algorithm. We show through our experiments that the selective use of incremental and full joins provides better performance than using one of them only.

A New Efficient Group-wise Spatial Multiplexing Design for Closed-Loop MIMO Systems (폐루프 다중입출력 시스템을 위한 효율적인 그룹별 공간 다중화 기법 설계)

  • Moon, Sung-Myun;Lee, Heun-Chul;Kim, Young-Tae;Lee, In-Kyu
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.4A
    • /
    • pp.322-331
    • /
    • 2010
  • This paper introduces a new efficient design scheme for spatial multiplexing (SM) systems over closed loop multiple-input multiple-output (MIMO) wireless channels. Extending the orthogonalized spatial multiplexing (OSM) scheme which was developed recently for transmitting two data streams, we propose a new SM scheme where a larger number of data streams can be supported. To achieve this goal, we partition the data streams into several subblocks and execute the block-diagonalization process at the receiver. The proposed scheme still guarantees single-symbol maximum likelihood (ML) detection with small feedback information. Simulation results verify that the proposed scheme achieves a huge performance gain at a bit error rate (BER) of $10^{-4}$ over conventional closed-loop schemes based on minimum mean-square error (MSE) or bit error rate (BER) criterion. We also show that an additional 2.5dB gain can be obtained by optimizing the group selection with extra feedback information.

GAGPC : An Algorithm to Optimize Multiple Continuous Queries on Data Streams (GAGPC : 데이타 스트림에 대한 다중 연속 질의의 최적화 알고리즘)

  • Suh Young-Kyoon;Son Jin-Hyun;Kim Myoung-Ho
    • Journal of KIISE:Databases
    • /
    • v.33 no.4
    • /
    • pp.409-422
    • /
    • 2006
  • In general, there can be many reusable intermediate results due to the overlapped windows and periodic execution intervals among Multiple Continuous Queries (MCQ) on data streams. In this regard, we propose an efficient greedy algorithm for a global query plan construction, called GAGPC. GAGPC first decides an execution cycle and finds the maximal Set(s) of Related execution Points (SRP). Next, GAGPC constructs a global execution plan to make MCQ share common join-fragments with the highest benefit in each SRP. The algorithm suggests that the best plan of the same continuous queries may be different according to not only the existence of common expressions, but the size of overlapped windows related to them. It also reflects to reuse not only the whole but partial intermediate results unlike previous work. Finally, we show experimental results for the validation of GAGPC.

Clustering based on Dependence Tree in Massive Data Streams

  • Yun, Hong-Won
    • Journal of information and communication convergence engineering
    • /
    • v.6 no.2
    • /
    • pp.182-186
    • /
    • 2008
  • RFID systems generate huge amount of data quickly. The data are associated with the locations and the timestamps and the containment relationships. It is requires to assure efficient queries and updates for product tracking and monitoring. We propose a clustering technique for fast query processing. Our study presents the state charts of temporal event flow and proposes the dependence trees with data association and uses them to cluster the linked events. Our experimental evaluation show the power of proposing clustering technique based on dependence tree.

Effective Load Shedding for Multi-Way windowed Joins Based on the Arrival Order of Tuples on Data Streams (다중 윈도우 조인을 위한 튜플의 도착 순서에 기반한 효과적인 부하 감소 기법)

  • Kwon, Tae-Hyung;Lee, Ki-Yong;Son, Jin-Hyun;Kim, Myoung-Ho
    • Journal of KIISE:Databases
    • /
    • v.37 no.1
    • /
    • pp.1-11
    • /
    • 2010
  • Recently, there has been a growing interest in the processing of continuous queries over multiple data streams. When the arrival rates of tuples exceed the memory capacity of the system, a load shedding technique is used to avoid the system becoming overloaded by dropping some subset of input tuples. In this paper, we propose an effective load shedding algorithm for multi-way windowed joins over multiple data streams. Most previous load shedding algorithms estimate the productivity of each tuple, i.e., the number of join output tuples produced by the tuple, based on its "join attribute value" and drop tuples with the lowest productivity. However, the productivity of a tuple cannot be accurately estimated from its join attribute value when the join attribute values are unique and do not repeat, or the distribution of the join attribute values changes over time. For these cases, we estimate the productivity of a tuple based on its "arrival order" on data streams, rather than its join attribute value. The proposed method can effectively estimate the productivity of a tuple even when the productivity of a tuple cannot be accurately estimated from its join attribute value. Through extensive experiments and analysis, we show that our proposed method outperforms the previous methods in terms of effectiveness and efficiency.