• Title/Summary/Keyword: Stream Processing

Search Result 720, Processing Time 0.026 seconds

CONTINUOUS QUERY PROCESSING IN A DATA STREAM ENVIRONMENT

  • Lee, Dong-Gyu;Lee, Bong-Jae;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2007.10a
    • /
    • pp.3-5
    • /
    • 2007
  • Many continuous queries are important to be process efficiently in a data stream environment. It is applied a query index technique that takes linear performance irrespective of the number and width of intervals for processing many continuous queries. Previous researches are not able to support the dynamic insertion and deletion to arrange intervals for constructing an index previously. It shows that the insertion and search performance is slowed by the number and width of interval inserted. Many intervals have to be inserted and searched linearly in a data stream environment. Therefore, we propose Hashed Multiple Lists in order to process continuous queries linearly. Proposed technique shows fast linear search performance. It can be utilized the systems applying a sensor network, and preprocessing technique of spatiotemporal data mining.

  • PDF

Jave based Embedded System Design and Implementation for Real-time Stream Data Processing (Java 기반 실시간 센서 데이터스트림처리 및 임베디드 시스템 구현)

  • Kim, Hyu-Chan;Ko, Wan-Ki;Park, Sang-Yeol
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.4 no.2
    • /
    • pp.1-12
    • /
    • 2008
  • Home network is a technology that provides possibilities of monitoring/controling/mutilating-recognition between optional home network machines in residences. Currently, home network or other networks like entertainment, residential electronic networks are jumbled together with heterogeneous networks in a rampaging condition. In a reality of high expectation for home networks system like the mutual application for various machines, we are required to have the unification technology for conveniences to satisfy expectations. This thesis reflects how to develop Java applications or mutual products based on convenient interfaces actually that process various sensors which create real time data stream in Java platform through Java based sensor data-stream processing embedded middleware design and realization in real time.

Efficient Processing of Multidimensional Vessel USN Stream Data using Clustering Hash Table (클러스터링 해쉬 테이블을 이용한 다차원 선박 USN 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Oh, Il-Whan;Lee, Seong-Ro
    • Journal of the Institute of Electronics Engineers of Korea SP
    • /
    • v.47 no.6
    • /
    • pp.137-145
    • /
    • 2010
  • Digital vessel have to accurate and efficient mange the digital data from various sensors in the digital vessel. But, In sensor network, it is difficult to transmit and analyze the entire stream data depending on limited networks, power and processor. Therefore it is suitable to use alternative stream data processing after classifying the continuous stream data. In this paper, We propose efficient processing method that arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and pre-clustering using multiple Support Vector Machine(SVM) algorithm and manage hash table to summarized information. Processing performance improve as store and search and memory using hash table and usage reduced so maintain hash table in memory. We obtained to efficient result that accuracy rate and processing performance of proposal method using 35,912 data sets.

A Data Processing Mechanism in Sensor Network Environment (센서 네트워크 환경에서의 데이터 처리 메커니즘)

  • Park, Dae-Hyun;Kim, Young-Jun;Lee, Jeong-Hoom;Chong, Il-Young
    • Proceedings of the IEEK Conference
    • /
    • 2007.07a
    • /
    • pp.133-134
    • /
    • 2007
  • The effective data processing mechanism in the sensor network means data stream model and real-time query processing model for real-time processing of stream data. This mechanism can improve satisfaction of users and reduce delay rate of data processing. In this paper, we explain the problem which is occurred when users need to search certain information among information of stream data and describe reduction model of delay rate according to data transmission.

  • PDF

Implementation of Real-time Data Stream Processing for Predictive Maintenance of Offshore Plants (해양플랜트의 예지보전을 위한 실시간 데이터 스트림 처리 구현)

  • Kim, Sung-Soo;Won, Jongho
    • Journal of KIISE
    • /
    • v.42 no.7
    • /
    • pp.840-845
    • /
    • 2015
  • In recent years, Big Data has been a topic of great interest for the production and operation work of offshore plants as well as for enterprise resource planning. The ability to predict future equipment performance based on historical results can be useful to shuttling assets to more productive areas. Specifically, a centrifugal compressor is one of the major piece of equipment in offshore plants. This machinery is very dangerous because it can explode due to failure, so it is necessary to monitor its performance in real time. In this paper, we present stream data processing architecture that can be used to compute the performance of the centrifugal compressor. Our system consists of two major components: a virtual tag stream generator and a real-time data stream manager. In order to provide scalability for our system, we exploit a parallel programming approach to use multi-core CPUs to process the massive amount of stream data. In addition, we provide experimental evidence that demonstrates improvements in the stream data processing for the centrifugal compressor.

Comparisions of stream activation mechanisms in computer based teleconferencing systems for low delay (지연 축소를 위한 컴퓨터 영상회의 시스템의 시트림 동작 구조 비교)

  • Lee, Gyeong-Hui;Kim, Du-Hyeon;Gang, Min-Gyu;Jeong, Chan-Geun
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.2
    • /
    • pp.363-376
    • /
    • 1997
  • In this paper, we present a hardware architecture and a sofrware architecture for cimputer based teleconferencing systems.And also we analyse stream adtivation mechanisms for them form the viewpoint of delay. MuX that is a multimedia I/O server provides various processing elements for data I/O, synchronization, interleaving and mixing.We describe methods to build teleconferencing systems with the elements and compares the technique using master click with the techniquie using self clock.In the plase of dta input.the technique using self click is berrer than the technique using master clock.When we generate interleved stream from audio and video stream and activate channel objects by periodic audio stream as activation clock, dealy from imput audio stream to imterleved stream is reduced but delay for video stream is not reduced as much as in the case of audio stream.

  • PDF

Design and Implementation of a USN Middleware for Context-Aware and Sensor Stream Mining

  • Jin, Cheng-Hao;Lee, Yang-Koo;Lee, Seong-Ho;Yun, Un-il;Ryu, Keun-Ho
    • Spatial Information Research
    • /
    • v.19 no.1
    • /
    • pp.127-133
    • /
    • 2011
  • Recently, with the advances in sensor techniques and net work computing, Ubiquitous Sensor Network (USN) has been received a lot of attentions from various communities. The sensor nodes distributed in the sensor network tend to continuously generate a large amount of data, which is called stream data. Sensor stream data arrives in an online manner so that it is characterized as high-speed, real-time and unbounded and it requires fast data processing to get the up-to-date results. The data stream has many application domains such as traffic analysis, physical distribution, U-healthcare and so on. Therefore, there is an overwhelming need of a USN middleware for processing such online stream data to provide corresponding services to diverse applications. In this paper, we propose a novel USN middleware which can provide users both context-aware service and meaningful sequential patterns. Our proposed USN middleware is mainly focused on location based applications which use stream location data. We also show the implementation of our proposed USN middleware. By using the proposed USN middleware, we can save the developing cost of providing context aware services and stream sequential patterns mainly in location based applications.

XML Fragmentation for Resource-Efficient Query Processing over XML Fragment Stream (자원 효율적인 XML 조각 스트림 질의 처리를 위한 XML 분할)

  • Kim, Jin;Kang, Hyun-Chul
    • The KIPS Transactions:PartD
    • /
    • v.16D no.1
    • /
    • pp.27-42
    • /
    • 2009
  • In realizing ubiquitous computing, techniques of efficiently using the limited resource at client such as mobile devices are required. With a mobile device with limited amount of memory, the techniques of XML stream query processing should be employed to process queries over a large volume of XML data. Recently, several techniques were proposed which fragment XML documents into XML fragments and stream them for query processing at client. During query processing, there could be great difference in resource usage (query processing time and memory usage) depending on how the source XML documents are fragmented. As such, an efficient fragmentation technique is needed. In this paper, we propose an XML fragmentation technique whereby resource efficiency in query processing at client could be enhanced. For this, we first present a cost model of query processing over XML fragment stream. Then, we propose an algorithm for resource-efficient XML fragmentation. Through implementation and experiments, we showed that our fragmentation technique outperformed previous techniques both in processing time and memory usage. The contribution of this paper is to have made the techniques of query processing over XML fragment stream more feasible for practical use.

FTSnet: A Simple Convolutional Neural Networks for Action Recognition (FTSnet: 동작 인식을 위한 간단한 합성곱 신경망)

  • Zhao, Yulan;Lee, Hyo Jong
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2021.11a
    • /
    • pp.878-879
    • /
    • 2021
  • Most state-of-the-art CNNs for action recognition are based on a two-stream architecture: RGB frames stream represents the appearance and the optical flow stream interprets the motion of action. However, the cost of optical flow computation is very high and then it increases action recognition latency. We introduce a design strategy for action recognition inspired by a two-stream network and teacher-student architecture. There are two sub-networks in our neural networks, the optical flow sub-network as a teacher and the RGB frames sub-network as a student. In the training stage, we distill the feature from the teacher as a baseline to train student sub-network. In the test stage, we only use the student so that the latency reduces without computing optical flow. Our experiments show that its advantages over two-stream architecture in both speed and performance.

Performance Optimization Considering I/O Data Coherency in Stream Processing (Stream Processing에서 I/O데이터 일관성을 고려한 성능 최적화)

  • Na, Hana;Yi, Joonwhan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.53 no.8
    • /
    • pp.59-65
    • /
    • 2016
  • Performance optimization of applications with massive stream data processing has been performed by considering I/O data coherency problem where a memory is shared between processors and hardware accelerators. A formula for performance analyses is derived based on profiling results of system-level simulations. Our experimental results show that overall performance was improved by 1.40 times on average for various image sizes. Also, further optimization has been performed based on the parameters appeared in the derived formula. The final performance gain was 3.88 times comparing to the original design and we can find that the performance of the design with cacheable shared memory is not always.