• Title/Summary/Keyword: Real-time Data Processing

Search Result 2,085, Processing Time 0.033 seconds

Adaptive Priority Queue-driven Task Scheduling for Sensor Data Processing in IoT Environments (사물인터넷 환경에서 센서데이터의 처리를 위한 적응형 우선순위 큐 기반의 작업 스케줄링)

  • Lee, Mijin;Lee, Jong Sik;Han, Young Shin
    • Journal of Korea Multimedia Society
    • /
    • v.20 no.9
    • /
    • pp.1559-1566
    • /
    • 2017
  • Recently in the IoT(Internet of Things) environment, a data collection in real-time through device's sensor has increased with an emergence of various devices. Collected data from IoT environment shows a large scale, non-uniform generation cycle and atypical. For this reason, the distributed processing technique is required to analyze the IoT sensor data. However if you do not consider the optimal scheduling for data and the processor of IoT in a distributed processing environment complexity increase the amount in assigning a task, the user is difficult to guarantee the QoS(Quality of Service) for the sensor data. In this paper, we propose APQTA(Adaptive Priority Queue-driven Task Allocation method for sensor data processing) to efficiently process the sensor data generated by the IoT environment. APQTA is to separate the data into job and by applying the priority allocation scheduling based on the deadline to ensure that guarantee the QoS at the same time increasing the efficiency of the data processing.

Development of a Sensor System to Measure Real Time Vibro Displacement of Civil Structure (레이저 센서를 이용한 구조물의 변위 측정 장비 개발)

  • O, Heung-Il;Kim, Hui-Sik
    • Proceedings of the KIEE Conference
    • /
    • 2003.11c
    • /
    • pp.823-825
    • /
    • 2003
  • A sensor system was designed to measure real time vibro displacement of civil structure. The He-Ne laser is used for the displacement measuring method, because it guarantees short time stabilization, long time output power stability. Also, it guarantees simple maintenances and repairs under actual using condition. The line CCD image sensor(Tcd-142d) is used to detect the displacement of Ne-Ne laser responding to the vibro of civil structure. For accurate measurement and comparison, CDP-50 is used. Usually CDF-50 (Strain type displacement device) is used for the standard correction device of optical measurement equipments. The data processing part is consists of Optical sensor part, Wireless data transmission device, DAQp-1200, and LapView program. The displacement data of vibro from optical sensor part inputted to wireless data transmission device and then transmitted to DAQp-1200 in main control room. DAQp-1200 performs A/D conversion for the receiving data. After that the converted data inputted to computer system using LapView program for user display. The significance of this paper is to develope a convenient, accurate and lost saving real time displacement measurement system for the civil structure.

  • PDF

Approximate Top-k Subgraph Matching Scheme Considering Data Reuse in Large Graph Stream Environments (대용량 그래프 스트림 환경에서 데이터 재사용을 고려한 근사 Top-k 서브 그래프 매칭 기법)

  • Choi, Do-Jin;Bok, Kyoung-Soo;Yoo, Jae-Soo
    • The Journal of the Korea Contents Association
    • /
    • v.20 no.8
    • /
    • pp.42-53
    • /
    • 2020
  • With the development of social network services, graph structures have been utilized to represent relationships among objects in various applications. Recently, a demand of subgraph matching in real-time graph streams has been increased. Therefore, an efficient approximate Top-k subgraph matching scheme for low latency in real-time graph streams is required. In this paper, we propose an approximate Top-k subgraph matching scheme considering data reuse in graph stream environments. The proposed scheme utilizes the distributed stream processing platform, called Storm to handle a large amount of stream data. We also utilize an existing data reuse scheme to decrease stream processing costs. We propose a distance based summary indexing technique to generate Top-k subgraph matching results. The proposed summary indexing technique costs very low since it only stores distances among vertices that are selected in advance. Finally, we provide k subgraph matching results to users by performing an approximate Top-k matching on the summary indexing. In order to show the superiority of the proposed scheme, we conduct various performance evaluations in diverse real world datasets.

Design and Analysis of MPEG-2 MP@HL Decoder in Multi-Processor Environments

  • Yoo, Seung-Hwan;Lee, Hyun-Seung;Lee, Sang-Jo;Park, Rae-Hong;Kim, Do-Hyung
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2009.01a
    • /
    • pp.211-216
    • /
    • 2009
  • As demands for high-definition television (HDTV) increase, the implementation of real-time decoding of high-definition (HD) video becomes an important issue. The data size for HD video is so large that real-time processing of the data is difficult to implement, especially with software. In order to implement a fast moving picture expert group-2 decoder for HDTV, we compose five scenarios that use parallel processing techniques such as data decomposition, task decomposition, and pipelining. Assuming the multi digital signal processor environments, we analyze each scenario in three aspects: decoding speed, L1 memory size, and bandwidth. By comparing the scenarios, we decide the most suitable cases for different situations. We simulate the scenarios in the dual-core and dual-central processing unit environment by using OpenMP and analyze the simulation results.

  • PDF

Real-time automated detection of construction noise sources based on convolutional neural networks

  • Jung, Seunghoon;Kang, Hyuna;Hong, Juwon;Hong, Taehoon;Lee, Minhyun;Kim, Jimin
    • International conference on construction engineering and project management
    • /
    • 2020.12a
    • /
    • pp.455-462
    • /
    • 2020
  • Noise which is unwanted sound is a serious pollutant that can affect human health, as well as the working and living environment if exposed to humans. However, current noise management on the construction project is generally conducted after the noise exceeds the regulation standard, which increases the conflicts with inhabitants near the construction site and threats to the safety and productivity of construction workers. To overcome the limitations of the current noise management methods, the activities of construction equipment which is the main source of construction noise need to be managed throughout the construction period in real-time. Therefore, this paper proposed a framework for automatically detecting noise sources in construction sites in real-time based on convolutional neural networks (CNNs) according to the following four steps: (i) Step 1: Definition of the noise sources; (ii) Step 2: Data preparation; (iii) Step 3: Noise source classification using the audio CNN; and (iv) Step 4: Noise source detection using the visual CNN. The short-time Fourier transform (STFT) and temporal image processing are used to contain temporal features of the audio and visual data. In addition, the AlexNet and You Only Look Once v3 (YOLOv3) algorithms have been adopted to classify and detect the noise sources in real-time. As a result, the proposed framework is expected to immediately find construction activities as current noise sources on the video of the construction site. The proposed framework could be helpful for environmental construction managers to efficiently identify and control the noise by automatically detecting the noise sources among many activities carried out by various types of construction equipment. Thereby, not only conflicts between inhabitants and construction companies caused by construction noise can be prevented, but also the noise-related health risks and productivity degradation for construction workers and inhabitants near the construction site can be minimized.

  • PDF

Design and Implementation of Event Hierarchy through Extended Spatio-Temporal Complex Event Processing (시공간 복합 이벤트 처리의 확장을 통한 계층적 이벤트 설계 및 구현)

  • Park, Ye Jin;Yom, Jae-Hong
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.30 no.6_1
    • /
    • pp.549-557
    • /
    • 2012
  • Spatial phenomena such as environment pollution, disease and the risk of spreading information need a rapid initial response to perceive spread event. Moving data perceive spread event through real-time processing and analysis. To process and analysis the event, spatial-temporal complex event processing is used. Previous spatialtemporal complex event processing is possible basis spatial operator but insufficient apply to design spatialtemporal complex event processing to perceive spatial phenomena of high complexity. This study proposed hierarchical spatio-temporal CEP design which will efficiently manage the fast growing incoming sensor data. The implementation of the proposed design is evaluated with GPS location data of moving vehicles which are used as the incoming data stream for identifying spatial events. The spatial component of existing CEP software engine has been extended during the implementation phase to broaden the capabilities of processing spatio-temporal events.

An Improved Skyline Query Scheme for Recommending Real-Time User Preference Data Based on Big Data Preprocessing (빅데이터 전처리 기반의 실시간 사용자 선호 데이터 추천을 위한 개선된 스카이라인 질의 기법)

  • Kim, JiHyun;Kim, Jongwan
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.11 no.5
    • /
    • pp.189-196
    • /
    • 2022
  • Skyline query is a scheme for exploring objects that are suitable for user preferences based on multiple attributes of objects. Existing skyline queries return search results as batch processing, but the need for real-time search results has increased with the advent of interactive apps or mobile environments. Online algorithm for Skyline improves the return speed of objects to explore preferred objects in real time. However, the object navigation process requires unnecessary navigation time due to repeated comparative operations. This paper proposes a Pre-processing Online Algorithm for Skyline Query (POA) to eliminate unnecessary search time in Online Algorithm exploration techniques and provide the results of skyline queries in real time. Proposed techniques use the concept of range-limiting to existing Online Algorithm to perform pretreatment and then eliminate repetitive rediscovering regions first. POAs showed improvement in standard distributions, bias distributions, positive correlations, and negative correlations of discrete data sets compared to Online Algorithm. The POAs used in this paper improve navigation performance by minimizing comparison targets for Online Algorithm, which will be a new criterion for rapid service to users in the face of increasing use of mobile devices.

Real-time Full-view 3D Human Reconstruction using Multiple RGB-D Cameras

  • Yoon, Bumsik;Choi, Kunwoo;Ra, Moonsu;Kim, Whoi-Yul
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.4 no.4
    • /
    • pp.224-230
    • /
    • 2015
  • This manuscript presents a real-time solution for 3D human body reconstruction with multiple RGB-D cameras. The proposed system uses four consumer RGB/Depth (RGB-D) cameras, each located at approximately $90^{\circ}$ from the next camera around a freely moving human body. A single mesh is constructed from the captured point clouds by iteratively removing the estimated overlapping regions from the boundary. A cell-based mesh construction algorithm is developed, recovering the 3D shape from various conditions, considering the direction of the camera and the mesh boundary. The proposed algorithm also allows problematic holes and/or occluded regions to be recovered from another view. Finally, calibrated RGB data is merged with the constructed mesh so it can be viewed from an arbitrary direction. The proposed algorithm is implemented with general-purpose computation on graphics processing unit (GPGPU) for real-time processing owing to its suitability for parallel processing.

Implementation of Wavelet Transform for a Real time Monitoring ECG Telemetry System (웨이브렛 변환을 이용한 실시간 모니터링 ECG 텔레미트리 시스템 구현)

  • 박차훈;서희돈
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.3 no.1
    • /
    • pp.27-32
    • /
    • 2002
  • In this study, we fabricated the advanced telemetry system that transmitting media use radio frequency(RF) for the middle range measurement of the physiological signals and receiving media use optical for electromagnetic interference problem. The telemetry system within a size of 65$\times$125$\times$45mm consists of three parts: RF transmitter, optical receiver and physiological signal processing CMOS one chip. Advantages of proposed telemetry system is wireless middle range(50m) FM transmission, reduce electromagnetic interference to a minimum which enables a comfortable bed-side telemetry system. The monitoring system was designed in the structure of dual-processor for the real time processing. The use of the one channel in our study made it possible the real time wavelet transformation of electrocardiogram data of 360Hz, 16 bits for every 1.42 seconds.

  • PDF

Performance Comparison of Guitar Chords Classification Systems Based on Artificial Neural Network (인공신경망 기반의 기타 코드 분류 시스템 성능 비교)

  • Park, Sun Bae;Yoo, Do-Sik
    • Journal of Korea Multimedia Society
    • /
    • v.21 no.3
    • /
    • pp.391-399
    • /
    • 2018
  • In this paper, we construct and compare various guitar chord classification systems using perceptron neural network and convolutional neural network without pre-processing other than Fourier transform to identify the optimal chord classification system. Conventional guitar chord classification schemes use, for better feature extraction, computationally demanding pre-processing techniques such as stochastic analysis employing a hidden markov model or an acoustic data filtering and hence are burdensome for real-time chord classifications. For this reason, we construct various perceptron neural networks and convolutional neural networks that use only Fourier tranform for data pre-processing and compare them with dataset obtained by playing an electric guitar. According to our comparison, convolutional neural networks provide optimal performance considering both chord classification acurracy and fast processing time. In particular, convolutional neural networks exhibit robust performance even when only small fraction of low frequency components of the data are used.