• Title/Summary/Keyword: Real-time Data Processing

Search Result 2,073, Processing Time 0.025 seconds

A Design and Implementation of Real-time Video frame data Processing control for Block Matching Algorithm (고속블럭정합 알고리즘을 위한 실시간 영상프레임 데이터 처리 제어 방법의 설계 및 구현)

  • 이강환;황호정
    • Proceedings of the IEEK Conference
    • /
    • 2001.06b
    • /
    • pp.373-376
    • /
    • 2001
  • This paper has been studied a real-time video frame data processing control that used the linear systolic array for motion estimation. The proposed data control processing provides to the input data into the multiple processor array unit(MPAU) from search area and reference block data. The proposed data control architecture has based on two slice band for input data processing. And it has no required external control logic blocks for input data as like reference block or search area data.

  • PDF

Real-Time IoT Big-data Processing for Stream Reasoning (스트림-리즈닝을 위한 실시간 사물인터넷 빅-데이터 처리)

  • Yun, Chang Ho;Park, Jong Won;Jung, Hae Sun;Lee, Yong Woo
    • Journal of Internet Computing and Services
    • /
    • v.18 no.3
    • /
    • pp.1-9
    • /
    • 2017
  • Smart Cities intelligently manage numerous infrastructures, including Smart-City IoT devices, and provide a variety of smart-city applications to citizen. In order to provide various information needed for smart-city applications, Smart Cities require a function to intelligently process large-scale streamed big data that are constantly generated from a large number of IoT devices. To provide smart services in Smart-City, the Smart-City Consortium uses stream reasoning. Our stream reasoning requires real-time processing of big data. However, there are limitations associated with real-time processing of large-scale streamed big data in Smart Cities. In this paper, we introduce one of our researches on cloud computing based real-time distributed-parallel-processing to be used in stream-reasoning of IoT big data in Smart Cities. The Smart-City Consortium introduced its previously developed smart-city middleware. In the research for this paper, we made cloud computing based real-time distributed-parallel-processing available in the cloud computing platform of the smart-city middleware developed in the previous research, so that we can perform real-time distributed-parallel-processing with them. This paper introduces a real-time distributed-parallel-processing method and system for stream reasoning with IoT big data transmitted from various sensors of Smart Cities and evaluate the performance of real-time distributed-parallel-processing of the system where the method is implemented.

EXCUTE REAL-TIME PROCESSING IN RTOS ON 8BIT MCU WITH TEMP AND HUMIDITY SENSOR

  • Kim, Ki-Su;Lee, Jong-Chan
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.11
    • /
    • pp.21-27
    • /
    • 2019
  • Recently, embedded systems have been introduced in various fields such as smart factories, industrial drones, and medical robots. Since sensor data collection and IoT functions for machine learning and big data processing are essential in embedded systems, it is essential to port the operating system that is suitable for the function requirements. However, in embedded systems, it is necessary to separate the hard real-time system, which must process within a fixed time according to service characteristics, and the flexible real-time system, which is more flexible in processing time. It is difficult to port the operating system to a low-performance embedded device such as 8BIT MCU to perform simultaneous real-time. When porting a real-time OS (RTOS) to a low-specification MCU and performing a number of tasks, the performance of the real-time and general processing greatly deteriorates, causing a problem of re-designing the hardware and software if a hard real-time system is required for an operating system ported to a low-performance MCU such as an 8BIT MCU. Research on the technology that can process real-time processing system requirements on RTOS (ported in low-performance MCU) is needed.

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

Real-time Event Processing Role Management System for IFTTT Service (IFTTT 서비스를 위한 실시간 이벤트 처리 룰 관리 시스템)

  • Kim, KyeYoung;Lee, HyunDong;Cho, Dae-Soo
    • Journal of Korea Multimedia Society
    • /
    • v.20 no.8
    • /
    • pp.1379-1386
    • /
    • 2017
  • As the Internet of Things evolves, various IoT services are provided. IFTTT is an abbreviation for If This Then That and refers to a service that links different web-based services. This paper proposes a system that generates and manages rules that combine the possibility of IFTTT service and the real-time event processing according to the concept of IoT service. Conventional database-based data processing methods are burdened to process a lot of data of IoT devices coming in real-time. The IoT device's data can be classified into formal data such as the amount of power, temperature value and position information, and informal data such as video or image data. Thus, this system classifies the data stream of IoT devices coming in real-time using the CEP engine Esper into a file signature table, classifies the formal/informal data, and shows the condition of the device data defined by the user and the service to be provided by applying the service.

Implementation of a Real-time Data fusion Algorithm for Flight Test Computer (비행시험통제컴퓨터용 실시간 데이터 융합 알고리듬의 구현)

  • Lee, Yong-Jae;Won, Jong-Hoon;Lee, Ja-Sung
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.8 no.4 s.23
    • /
    • pp.24-31
    • /
    • 2005
  • This paper presents an implementation of a real-time multi-sensor data fusion algorithm for Flight Test Computer. The sensor data consist of positional information of the target from a radar, a GPS receiver and an INS. The data fusion algorithm is designed by the 21st order distributed Kalman Filter which is based on the PVA model with sensor bias states. A fault detection and correction logics are included in the algorithm for bad measurements and sensor faults. The statistical parameters for the states are obtained from Monte Carlo simulations and covariance analysis using test tracking data. The designed filter is verified by using real data both in post processing and real-time processing.

A Method of Data Transmission for Performance Improvement of Real Time GNSS Data Processing in Multi-Reference Network Station (다중 수신국 실시간 위성항법데이터 처리 성능향상을 위한 데이터 송·수신 설계)

  • Kim, Gue-Heon;Son, Minhyuk;Lee, Eunsung;Heo, Moon-Beom
    • Journal of the Korean Society for Aviation and Aeronautics
    • /
    • v.20 no.4
    • /
    • pp.39-44
    • /
    • 2012
  • This paper propose a transmission method for "Transportation system" that can decide precise position under wide area road traffic environment. For precise position detecting, central station collect multiple receiver station's satellite navigation data and generate correction information. In this process, we need efficient real time transmission method for satellite navigation message that has variable data size. We propose real time data transmission method. This real time transmission method offer efficient processing structure for multiple receiver station's satellite navigation message. This paper explains proposed real time transmission method and proofs this transmission method.

Unmanned Aircraft Platform Based Real-time LiDAR Data Processing Architecture for Real-time Detection Information (실시간 탐지정보 제공을 위한 무인기 플랫폼 기반 실시간 LiDAR 데이터 처리구조)

  • Eum, Junho;Berhanu, Eyassu;Oh, Sangyoon
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.12
    • /
    • pp.745-750
    • /
    • 2015
  • LiDAR(Light Detection and Ranging) technology provides realistic 3-dimension image information, and it has been widely utilized in various fields. However, the utilization of this technology in the military domain requires prompt responses to dynamically changing tactical environment and is therefore limited since LiDAR technology requires complex processing in order for extensive amounts of LiDAR data to be utilized. In this paper, we introduce an Unmanned Aircraft Platform Based Real-time LiDAR Data Processing Architecture that can provide real-time detection information by parallel processing and off-loading between the UAV processing and high-performance data processing areas. We also conducted experiments to verify the feasibility of our proposed architecture. Processing with ARM cluster similar to the UAV platform processing area results in similar or better performance when compared to the current method. We determined that our proposed architecture can be utilized in the military domain for tactical and combat purposes such as unmanned monitoring system.

Comparison of different post-processing techniques in real-time forecast skill improvement

  • Jabbari, Aida;Bae, Deg-Hyo
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2018.05a
    • /
    • pp.150-150
    • /
    • 2018
  • The Numerical Weather Prediction (NWP) models provide information for weather forecasts. The highly nonlinear and complex interactions in the atmosphere are simplified in meteorological models through approximations and parameterization. Therefore, the simplifications may lead to biases and errors in model results. Although the models have improved over time, the biased outputs of these models are still a matter of concern in meteorological and hydrological studies. Thus, bias removal is an essential step prior to using outputs of atmospheric models. The main idea of statistical bias correction methods is to develop a statistical relationship between modeled and observed variables over the same historical period. The Model Output Statistics (MOS) would be desirable to better match the real time forecast data with observation records. Statistical post-processing methods relate model outputs to the observed values at the sites of interest. In this study three methods are used to remove the possible biases of the real-time outputs of the Weather Research and Forecast (WRF) model in Imjin basin (North and South Korea). The post-processing techniques include the Linear Regression (LR), Linear Scaling (LS) and Power Scaling (PS) methods. The MOS techniques used in this study include three main steps: preprocessing of the historical data in training set, development of the equations, and application of the equations for the validation set. The expected results show the accuracy improvement of the real-time forecast data before and after bias correction. The comparison of the different methods will clarify the best method for the purpose of the forecast skill enhancement in a real-time case study.

  • PDF

A Real-Time Integrated Hierarchical Temporal Memory Network for the Real-Time Continuous Multi-Interval Prediction of Data Streams

  • Kang, Hyun-Syug
    • Journal of Information Processing Systems
    • /
    • v.11 no.1
    • /
    • pp.39-56
    • /
    • 2015
  • Continuous multi-interval prediction (CMIP) is used to continuously predict the trend of a data stream based on various intervals simultaneously. The continuous integrated hierarchical temporal memory (CIHTM) network performs well in CMIP. However, it is not suitable for CMIP in real-time mode, especially when the number of prediction intervals is increased. In this paper, we propose a real-time integrated hierarchical temporal memory (RIHTM) network by introducing a new type of node, which is called a Zeta1FirstSpecializedQueueNode (ZFSQNode), for the real-time continuous multi-interval prediction (RCMIP) of data streams. The ZFSQNode is constructed by using a specialized circular queue (sQUEUE) together with the modules of original hierarchical temporal memory (HTM) nodes. By using a simple structure and the easy operation characteristics of the sQUEUE, entire prediction operations are integrated in the ZFSQNode. In particular, we employed only one ZFSQNode in each level of the RIHTM network during the prediction stage to generate different intervals of prediction results. The RIHTM network efficiently reduces the response time. Our performance evaluation showed that the RIHTM was satisfied to continuously predict the trend of data streams with multi-intervals in the real-time mode.