• Title/Summary/Keyword: Data Processing Time

Search Result 5,280, Processing Time 0.038 seconds

Adaptive Priority Queue-driven Task Scheduling for Sensor Data Processing in IoT Environments (사물인터넷 환경에서 센서데이터의 처리를 위한 적응형 우선순위 큐 기반의 작업 스케줄링)

  • Lee, Mijin;Lee, Jong Sik;Han, Young Shin
    • Journal of Korea Multimedia Society
    • /
    • v.20 no.9
    • /
    • pp.1559-1566
    • /
    • 2017
  • Recently in the IoT(Internet of Things) environment, a data collection in real-time through device's sensor has increased with an emergence of various devices. Collected data from IoT environment shows a large scale, non-uniform generation cycle and atypical. For this reason, the distributed processing technique is required to analyze the IoT sensor data. However if you do not consider the optimal scheduling for data and the processor of IoT in a distributed processing environment complexity increase the amount in assigning a task, the user is difficult to guarantee the QoS(Quality of Service) for the sensor data. In this paper, we propose APQTA(Adaptive Priority Queue-driven Task Allocation method for sensor data processing) to efficiently process the sensor data generated by the IoT environment. APQTA is to separate the data into job and by applying the priority allocation scheduling based on the deadline to ensure that guarantee the QoS at the same time increasing the efficiency of the data processing.

Designing Cost Effective Open Source System for Bigdata Analysis (빅데이터 분석을 위한 비용효과적 오픈 소스 시스템 설계)

  • Lee, Jong-Hwa;Lee, Hyun-Kyu
    • Knowledge Management Research
    • /
    • v.19 no.1
    • /
    • pp.119-132
    • /
    • 2018
  • Many advanced products and services are emerging in the market thanks to data-based technologies such as Internet (IoT), Big Data, and AI. The construction of a system for data processing under the IoT network environment is not simple in configuration, and has a lot of restrictions due to a high cost for constructing a high performance server environment. Therefore, in this paper, we will design a development environment for large data analysis computing platform using open source with low cost and practicality. Therefore, this study intends to implement a big data processing system using Raspberry Pi, an ultra-small PC environment, and open source API. This big data processing system includes building a portable server system, building a web server for web mining, developing Python IDE classes for crawling, and developing R Libraries for NLP and visualization. Through this research, we will develop a web environment that can control real-time data collection and analysis of web media in a mobile environment and present it as a curriculum for non-IT specialists.

The Development of the Real Time Target Simulator for the RF Signal of Electronic Warfare using VST and FPGA (VST 및 FPGA를 이용한 전자표적 생성 및 신호 모의장치 개발)

  • Sanghun Song
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.26 no.4
    • /
    • pp.324-334
    • /
    • 2023
  • In this paper, the target simulator for RF signals was developed by using VST(Vector Signal Transceiver) and set by real-time signal processing SW programs. A function to process RF signals using FPGA(Field Programmable Gate Array) board was designed. The system functions capable of data processing, raw signals monitoring, target signals(simulated range, velocity) generating and RF environments data analyzing were implemented. And the characteristics of modulated signal were analyzed in RF environment. All function of programs for processing RF signal have options to store signal data and to manage the data. The validity of the signal simulation was confirmed through verification of simulated signal results.

Real-time Audio Processing for TCP/IP in Server-Client Model (서버-클라이언트 모델에서의 TCP/IP 기반 실시간 음성 처리)

  • Lee, Hyung-ho;Jeong, Dae-young;Park, Kyung-tae;You, Byung-sek;Kim, Jeong-sig
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2013.10a
    • /
    • pp.619-621
    • /
    • 2013
  • This paper is proposing a real-time audio processing system for TCP/IP with server-client. The server sends the audio data packet which is the same size each time while playing the audio data. And the client plays the received audio data from the server. In general, The receiving speed of audio data packet is faster than processing the audio data. So, the unstable playback is occurred when playing the received audio data at the moment. In order to overcome this problem, the double buffering method is proposed.

  • PDF

An Adaptive Workflow Scheduling Scheme Based on an Estimated Data Processing Rate for Next Generation Sequencing in Cloud Computing

  • Kim, Byungsang;Youn, Chan-Hyun;Park, Yong-Sung;Lee, Yonggyu;Choi, Wan
    • Journal of Information Processing Systems
    • /
    • v.8 no.4
    • /
    • pp.555-566
    • /
    • 2012
  • The cloud environment makes it possible to analyze large data sets in a scalable computing infrastructure. In the bioinformatics field, the applications are composed of the complex workflow tasks, which require huge data storage as well as a computing-intensive parallel workload. Many approaches have been introduced in distributed solutions. However, they focus on static resource provisioning with a batch-processing scheme in a local computing farm and data storage. In the case of a large-scale workflow system, it is inevitable and valuable to outsource the entire or a part of their tasks to public clouds for reducing resource costs. The problems, however, occurred at the transfer time for huge dataset as well as there being an unbalanced completion time of different problem sizes. In this paper, we propose an adaptive resource-provisioning scheme that includes run-time data distribution and collection services for hiding the data transfer time. The proposed adaptive resource-provisioning scheme optimizes the allocation ratio of computing elements to the different datasets in order to minimize the total makespan under resource constraints. We conducted the experiments with a well-known sequence alignment algorithm and the results showed that the proposed scheme is efficient for the cloud environment.

DEVELOPMENT OF ROI PROCESSING SYSTEM USING QUICK LOOK IMAGE

  • Ahn, Sang-Il;Kim, Tae-Hoon;Kim, Tae-Young;Koo, In-Hoi
    • Proceedings of the KSRS Conference
    • /
    • 2007.10a
    • /
    • pp.526-529
    • /
    • 2007
  • Due to its inherent feature of high-resolution satellite, there is strong need in some specific area to minimize the processing time required to get a standard image on hand from downlink signal acquisition. However, in general image processing system, it takes considerable time to get image data up to certain level from raw data acquisition because the huge amount of data is dealt sequentially as input data. This paper introduces the high-speed image processing system which generates the image data only for the area selected by user. To achieve the high speed performance, this system includes Quick Look Image display function with sampling, ROI selection function, Image Line Index function, and Distributed processing function. The developed RPS was applied to KOMPSAT-2 320Mbps downlink channel and its effectiveness was successfully demonstrated. This feature to provide the image product very quickly is expected to promote the application of high resolution satellite image.

  • PDF

Efficiency Low-Power Signal Processing for Multi-Channel LiDAR Sensor-Based Vehicle Detection Platform (멀티채널 LiDAR 센서 기반 차량 검출 플랫폼을 위한 효율적인 저전력 신호처리 기법)

  • Chong, Taewon;Park, Daejin
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.7
    • /
    • pp.977-985
    • /
    • 2021
  • The LiDAR sensor is attracting attention as a key sensor for autonomous driving vehicle. LiDAR sensor provides measured three-dimensional lengths within range using LASER. However, as much data is provided to the external system, it is difficult to process such data in an external system or processor of the vehicle. To resolve these issues, we develop integrated processing system for LiDAR sensor. The system is configured that client receives data from LiDAR sensor and processes data, server gathers data from clients and transmits integrated data in real-time. The test was carried out to ensure real-time processing of the system by changing the data acquisition, processing method and process driving method of process. As a result of the experiment, when receiving data from four LiDAR sensors, client and server process was operated using background or multi-core processing, the system response time of each client was about 13.2 ms and the server was about 12.6 ms.

A High-rate GPS Data Processing for Large-scale Structure Monitoring (대형구조물 모니터링을 위한 high-rate GPS 자료처리)

  • Bae, Tea-Suk
    • Proceedings of the Korean Society of Surveying, Geodesy, Photogrammetry, and Cartography Conference
    • /
    • 2010.04a
    • /
    • pp.181-182
    • /
    • 2010
  • For real-time displacement monitoring of large-scale structures, the high-rate (>1 Hz) GPS data processing is necessary, which is not possible even for the scientific GPS data processing softwares. Since the baseline is generally very short in this case, most of the atmospheric effects are removed, resulting in the unknowns of position and integer ambiguity. The number of unknowns in real-time kinematic GPS positioning makes the positioning impossible with usual approach, thus two-step approach is tested in this study.

  • PDF

PERFORMANCE EVALUATION AND DEVELOPMENT OF RVDB SYSTEM FOR THE SYNCHRONIZED PLAYBACK PROCESSING OF OBSERVED DATA IN KJJVC (한일공동VLBI상관기에서 관측 데이터의 동기재생처리를 위한 RVDB 시스템 개발과 성능시험)

  • Oh, Se-Jin;Roh, Duk-Gyoo;Yeom, Jae-Hwan;Chung, Hyun-Soo;Lee, Chang-Hoon;Kim, Kwang-Dong;Kim, Hyo-Ryoung;Oyama, Tomoaki;Kawaguchi, Noriyuki;Ozeki, Kensuke
    • Publications of The Korean Astronomical Society
    • /
    • v.23 no.2
    • /
    • pp.91-107
    • /
    • 2008
  • In this paper, we introduce the performance evaluation and development of Raw VLBI Data Buffer(RVDB) system for the synchronized playback processing of observed data in Korea-Japan Joint VLBI Correlator(KJJVC). The high-speed correlation processing system is under development so that the radio data obtained with 8192 channels and 8 Gbps speed from 16 stations will be able to be processed. When the recorded data of each station are played to correlator, the time synchronization of each station is very important because the correlator should process the data obtained with same time and condition. There are many types of recorder systems in the East Asia VLBI Network (EAVN). Therefore it is required to prepare the special time synchronized playback processing system to synchronize the time tag of observed data. The developed RVDB system consists of Data Input Output(DIO), 10GbE switch, and Disk Data Buffer(DDB). It can record the data with maximum 2 Gbps speed, and can play back the data to correlator with nominal 2 Gbps speed. To enable to play back the data of different playback system to the correlator, we developed the high-speed time synchronized playback processing system. We carried out the experiments of playing back and correlation for gigabit correlator and VCS trial product so as to confirm the performance of developed time synchronized playback processing system. In case of online and offline playing back experiment for gigabit correlator, we confirmed that the online and offline correlation results were the same. In case of playing back experiment for VCS trial product, we verified that the wide band and narrow band correlation results were also the same. Through the playing back experiments of RVDB system, the effectiveness of developed RVDB system was verified. In this paper, the system design, construction and experimental results are shown briefly.

Spatio-temporal Sensor Data Processing Techniques

  • Kim, Jeong-Joon
    • Journal of Information Processing Systems
    • /
    • v.13 no.5
    • /
    • pp.1259-1276
    • /
    • 2017
  • As technologies related to sensor network are currently emerging and the use of GeoSensor is increasing along with the development of Internet of Things (IoT) technology, spatial query processing systems to efficiently process spatial sensor data are being actively studied. However, existing spatial query processing systems do not support a spatial-temporal data type and a spatial-temporal operator for processing spatialtemporal sensor data. Therefore, they are inadequate for processing spatial-temporal sensor data like GeoSensor. Accordingly, this paper developed a spatial-temporal query processing system, for efficient spatial-temporal query processing of spatial-temporal sensor data in a sensor network. Lastly, this paper verified the utility of System through a scenario, and proved that this system's performance is better than existing systems through performance assessment of performance time and memory usage.