• Title/Summary/Keyword: stream processing

Search Result 720, Processing Time 0.032 seconds

Efficient Labeling Scheme for Query Processing over XML Fragment Stream in Wireless Computing (무선 환경에서 XML 조각 스트림 질의 처리를 위한 효율적인 레이블링 기법)

  • Ko, Hye-Kyeong
    • The KIPS Transactions:PartD
    • /
    • v.17D no.5
    • /
    • pp.353-358
    • /
    • 2010
  • Unlike the traditional databases, queries on XML streams are restricted to a real time processing and memory usage. In this paper, a robust labeling scheme is proposed, which quickly identifies structural relationship between XML fragments. The proposed labeling scheme provides an effective query processing by removing many redundant operations and minimizing the number of fragments being processed. In experimental results, the proposed labeling scheme efficiently processes query processing and optimizes memory usage.

A Method of Frequent Structure Detection Based on Active Sliding Window (능동적 슬라이딩 윈도우 기반 빈발구조 탐색 기법)

  • Hwang, Jeong-Hee
    • Journal of Digital Contents Society
    • /
    • v.13 no.1
    • /
    • pp.21-29
    • /
    • 2012
  • In ubiquitous computing environment, rising large scale data exchange through sensor network with sharply growing the internet, the processing of the continuous stream data is required. Therefore there are some mining researches related to the extracting of frequent structures and the efficient query processing of XML stream data. In this paper, we propose a mining method to extract frequent structures of XML stream data in recent window based on the active window sliding using trigger rule. The proposed method is a basic research to control the stream data flow for data mining and continuous query by trigger rules.

Stream Data Processing based on Sliding Window at u-Health System (u-Health 시스템에서 슬라이딩 윈도우 기반 스트림 데이터 처리)

  • Kim, Tae-Yeun;Song, Byoung-Ho;Bae, Sang-Hyun
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.4 no.2
    • /
    • pp.103-110
    • /
    • 2011
  • It is necessary to accurate and efficient management for measured digital data from sensors in u-health system. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using backpropagation algorithm. As a result, we obtained to efficient result about 18.3% reduction rate of database using 14,324 data sets.

Stream-based Biomedical Classification Algorithms for Analyzing Biosignals

  • Fong, Simon;Hang, Yang;Mohammed, Sabah;Fiaidhi, Jinan
    • Journal of Information Processing Systems
    • /
    • v.7 no.4
    • /
    • pp.717-732
    • /
    • 2011
  • Classification in biomedical applications is an important task that predicts or classifies an outcome based on a given set of input variables such as diagnostic tests or the symptoms of a patient. Traditionally the classification algorithms would have to digest a stationary set of historical data in order to train up a decision-tree model and the learned model could then be used for testing new samples. However, a new breed of classification called stream-based classification can handle continuous data streams, which are ever evolving, unbound, and unstructured, for instance--biosignal live feeds. These emerging algorithms can potentially be used for real-time classification over biosignal data streams like EEG and ECG, etc. This paper presents a pioneer effort that studies the feasibility of classification algorithms for analyzing biosignals in the forms of infinite data streams. First, a performance comparison is made between traditional and stream-based classification. The results show that accuracy declines intermittently for traditional classification due to the requirement of model re-learning as new data arrives. Second, we show by a simulation that biosignal data streams can be processed with a satisfactory level of performance in terms of accuracy, memory requirement, and speed, by using a collection of stream-mining algorithms called Optimized Very Fast Decision Trees. The algorithms can effectively serve as a corner-stone technology for real-time classification in future biomedical applications.

Aural-visual two-stream based infant cry recognition (Aural-visual two-stream 기반의 아기 울음소리 식별)

  • Bo, Zhao;Lee, Jonguk;Atif, Othmane;Park, Daihee;Chung, Yongwha
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2021.05a
    • /
    • pp.354-357
    • /
    • 2021
  • Infants communicate their feelings and needs to the outside world through non-verbal methods such as crying and displaying diverse facial expressions. However, inexperienced parents tend to decode these non-verbal messages incorrectly and take inappropriate actions, which might affect the bonding they build with their babies and the cognitive development of the newborns. In this paper, we propose an aural-visual two-stream based infant cry recognition system to help parents comprehend the feelings and needs of crying babies. The proposed system first extracts the features from the pre-processed audio and video data by using the VGGish model and 3D-CNN model respectively, fuses the extracted features using a fully connected layer, and finally applies a SoftMax function to classify the fused features and recognize the corresponding type of cry. The experimental results show that the proposed system classification exceeds 0.92 in F1-score, which is 0.08 and 0.10 higher than the single-stream aural model and single-stream visual model.

A Push-Caching and a Transmission Scheme of Continuous Media for NOD Service on the Internet (인테넷상에서 NOD 서비스를 위한 연속미디어 전송 및 푸쉬-캐싱 기법)

  • Park, Seong-Ho;Im, Eun-Ji;Choe, Tae-Uk;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.6
    • /
    • pp.1766-1777
    • /
    • 2000
  • In multimedia new service on the internet, there are problems such as server overload, network congestion and initial latency. To overcome these problems, we propose a proxy push-caching scheme that stores a portion of continuous media stream or entire stream, and a transmission scheme of NOD continuous media, RTP-RR and RTP-nR to exploit push-caching scheme. With the proposed push-caching scheme, NOD server pushes fixed portion of stream to a proxy when new data is generated, and the cached size of each stream changes dynamically according to the caching utility value of each stream. As a result, the initial latency of client side could be reduced and the amount of data transmitted fro ma proxy server to client could be increased. Moreover, we estimate a caching utility value of each stream using correlation between disk space occupied by the stream and the amount of data stream requested by client. And we applied the caching utility value ot replacement policies. The performance of the proxy push-caching and continuous media transmission schemes proposed were compared with other schemes using simulations. In the simulation, these schemes show better results than other schemes in terms of BHR (Byte Hit Rate), initial latency, the number of replacement and packet loss rate.

  • PDF

Load Balancing for Distributed Processing of Real-time Spatial Big Data Stream (실시간 공간 빅데이터 스트림 분산 처리를 위한 부하 균형화 방법)

  • Yoon, Susik;Lee, Jae-Gil
    • Journal of KIISE
    • /
    • v.44 no.11
    • /
    • pp.1209-1218
    • /
    • 2017
  • A variety of sensors is widely used these days, and it has become much easier to acquire spatial big data streams from various sources. Since spatial data streams have inherently skewed and dynamically changing distributions, the system must effectively distribute the load among workers. Previous studies to solve this load imbalance problem are not directly applicable to processing spatial data. In this research, we propose Adaptive Spatial Key Grouping (ASKG). The main idea of ASKG is, by utilizing the previous distribution of the data streams, to adaptively suggest a new grouping scheme that evenly distributes the future load among workers. We evaluate the validity of the proposed algorithm in various environments, by conducting an experiment with real datasets while varying the number of workers, input rate, and processing overhead. Compared to two other alternative algorithms, ASKG improves the system performance in terms of load imbalance, throughput, and latency.

Partition-based Operator Sharing Scheme for Spatio-temporal Data Stream Processing (시공간 데이터 스트림 처리를 위한 영역 기반의 연산자 공유 기법)

  • Chung, Weon-Il;Kim, Young-Ki
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.11 no.12
    • /
    • pp.5042-5048
    • /
    • 2010
  • In ubiquitous environments, many continuous query processing techniques make use of operator network and sharing methods on continuous data stream generated from various sensors. Since similar continuous queries with the location information intensively occur in specific regions, we suggest a new operator sharing method based on grid partition for the spatial continuous query processing for location-based applications. Due to the proposed method shares moving objects by the given grid cell without sharing spatial operators individually, our approach can not only share spatial operators including similar conditions, but also increase the query processing performance and the utilization of memory by reducing the frequency of use of spatial operators.

In-Memory Based Incremental Processing Method for Stream Query Processing in Big Data Environments (빅데이터 환경에서 스트림 질의 처리를 위한 인메모리 기반 점진적 처리 기법)

  • Bok, Kyoungsoo;Yook, Misun;Noh, Yeonwoo;Han, Jieun;Kim, Yeonwoo;Lim, Jongtae;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.16 no.2
    • /
    • pp.163-173
    • /
    • 2016
  • Recently, massive amounts of stream data have been studied for distributed processing. In this paper, we propose an incremental stream data processing method based on in-memory in big data environments. The proposed method stores input data in a temporary queue and compare them with data in a master node. If the data is in the master node, the proposed method reuses the previous processing results located in the node chosen by the master node. If there are no previous results of data in the node, the proposed method processes the data and stores the result in a separate node. We also propose a job scheduling technique considering the load and performance of a node. In order to show the superiority of the proposed method, we compare it with the existing method in terms of query processing time. Our experimental results show that our method outperforms the existing method in terms of query processing time.

A Study on the Traffic Flow Analysis Method by Image Processing (화상처리에 의한 교통류 해석방법에 관한 연구)

  • 이종달;이령욱
    • Journal of Korean Society of Transportation
    • /
    • v.12 no.1
    • /
    • pp.97-116
    • /
    • 1994
  • Today advanced traffic management systems are required because of a high increase in traffic demand. Accordingly, the objective of this study is to take advantage of image processing systems and present image processing methods available for collection of the data on traffic characteristics, and then to investigate the possibility of traffic flow analysis by means of comparison and analysis of measured traffic flow. Data were collected at two places of Daegu city and Kyongbu expressway by using VTR. Rear view (down stream) and frontal view (up stream) methods were employed to compare and analyze traffic characteristics including traffic volume, speed, time-headway, time-occupancy, and vehicle-length, by analysis of measured traffic flow and image processing respectively. Judging from the results obtained by this study, image processing techniques are sufficient for the analysis of traffic volume, but a frame grabber equipped with high speed processor is necessary as well, with low level system judged to be sufficient for traffic volume analysis.

  • PDF