• Title/Summary/Keyword: Data Processing Time

Search Result 5,280, Processing Time 0.034 seconds

QoS Analysis of a Distributed System Considering the Processing Time (처리시간을 고려한 분산시스템의 서비스 품질분석)

  • Kim, Jung-Ho;Park, Jong-Hun
    • Journal of Korean Society for Quality Management
    • /
    • v.39 no.3
    • /
    • pp.412-421
    • /
    • 2011
  • In this paper, we introduce Quality of Service(QoS) analytic model of a distributed system that decentralizes the process nodes performing each task and communicates through a network for cooperation. The model advances a service reliability model of Dai et a1.(2003) by means of considering the processing time. The service is assumed to be provided by a centralized heterogeneous distributed system which is composed of some subsystems managed by a control center. The QoS is defined as the probability that a service is provided successfully in an allowed time, we consider the hardware/software reliability and the processing time which include program execution time, data transfer time. We derive the processing time distribution for a required service through convolution of corresponding probability density function. An application example is used to explain the procedure of computing quality of service.

Design of High-Speed Image Processing System for Line-Scan Camera (라인 스캔 카메라를 위한 고속 영상 처리 시스템 설계)

  • 이운근;백광렬;조석빈
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.10 no.2
    • /
    • pp.178-184
    • /
    • 2004
  • In this paper, we designed an image processing system for the high speed line-scan camera which adopts the new memory model we proposed. As a resolution and a data rate of the line-scan camera are becoming higher, the faster image processing systems are needed. But many conventional systems are not sufficient to process the image data from the line-scan camera during a very short time. We designed the memory controller which eliminates the time for transferring image data from the line-scan camera to the main memory with high-speed SRAM and has a dual-port configuration therefore the DSP can access the main memory even though the memory controller are writing the image data. The memory controller is implemented by VHDL and Xilinx SPARTAN-IIE FPGA.

A Study on the Sonar Data Processing by Using a Discrete Wavelet Transform (이산 웨이브릿 변환을 이용한 소나 자료처리에 관한 연구)

  • Kim, Jin-Hoo;Kim, Hyun-Do
    • Proceedings of the Korea Committee for Ocean Resources and Engineering Conference
    • /
    • 2003.05a
    • /
    • pp.324-329
    • /
    • 2003
  • Spectral analysis is an important signal processing tool for time series data. The transformation of a time series into the frequency domain is the basis for a significant number of processing algorithms and interpretive methods. Recently developed transforms based on the new mathematical field of wavelet analysis bypass the resolution limitation and offer superior spectral decomposition. The discrete wavelet transform of Sonar data provides spectral localization of noises, hence noises can be filtered out successfully.

  • PDF

An Estimation Scheme on Processing Time and Processor Utilization for Real-Time System Development (실시간 시스템 개발을 위한 데이터 처리 시간과 프로세서 사용율 추정 기법)

  • Kim, Han-Dong;Choi, Tae-Bong;Ko, Soon-Ju
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2005.07a
    • /
    • pp.820-822
    • /
    • 2005
  • The current paper is on a study of the performance estimation fer data processing time and CPU utilization to efficiently develop the real-time system. The analytical modeling and OPNET modeling and benchmarking tests are applied to perform the estimation for data processing time and CPU utilization in real-time system. We demonstrate that the estimation results can be predicted fairly and accurately through the benchmarking test results although there is a small variance between the estimation results and the benchmarking test results.

  • PDF

Construction of Expert Service for GPS Relative Positioning Data Processing (GPS 상대측위 자료처리를 위한 전문가 서비스 구축)

  • Park, Joon-Kyu;Kim, Min-Gyu;Lee, Jong-Sin
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.5
    • /
    • pp.2481-2486
    • /
    • 2013
  • It requires a lot of time and effort for general users who do not have enough understanding of GPS to properly processing GPS data. However, the GPS data processing field heavily relies on foreign-produced software and there is almost no development of user-oriented technology. Therefore, in this study, it was attempted to build an expert service that enables non-experts to use high-precision GPS data processing. As a result, an expert service that can maximize user convenience simply by entering the minimum required information for GPS data processing was developed, and the expert service was verified by relative positioning processing of the observation data of satellite control point provided by National Geographic Information Institute and observation data obtained by GPS survey. The expert service significantly reduces the effort and time for processing GPS data, which will contribute to precise positioning and other various studies.

Time-Deterministic Event Processing in Terabit Optical-Circuit-Packet Converged Switching Systems (테라비트 광-회선-패킷 통합 스위칭 시스템에서 시간결정성 높은 이벤트 처리에 관한 연구)

  • Kim, Bup-Joong;Ryoo, Jeong-dong;Cho, Kyoungrok
    • Korean Journal of Optics and Photonics
    • /
    • v.27 no.6
    • /
    • pp.212-217
    • /
    • 2016
  • In connection-oriented data-transport services, data loss can occur when the service experiences a problem on its end-to-end path. To promptly resolve this problem, the data-switching systems providing the service should quickly modify their internal configurations distributed among different places in each system. This is performed through a sequence of problem (event) recognition, sharing, and handling procedures among multiple control processors in the system. This paper proposes a method for event sharing and messaging between control processors, to improve the time determinacy of event processing. This method simplifies runtime event sharing and minimizes the time variability caused by the event data, resulting in a decrease in the latency time in processing global events. The proposed method lessens the latency time of global event processing by about 40%, compared to general methods, for 738 internal path changes.

Time Series Data Cleaning Method Based on Optimized ELM Prediction Constraints

  • Guohui Ding;Yueyi Zhu;Chenyang Li;Jinwei Wang;Ru Wei;Zhaoyu Liu
    • Journal of Information Processing Systems
    • /
    • v.19 no.2
    • /
    • pp.149-163
    • /
    • 2023
  • Affected by external factors, errors in time series data collected by sensors are common. Using the traditional method of constraining the speed change rate to clean the errors can get good performance. However, they are only limited to the data of stable changing speed because of fixed constraint rules. Actually, data with uneven changing speed is common in practice. To solve this problem, an online cleaning algorithm for time series data based on dynamic speed change rate constraints is proposed in this paper. Since time series data usually changes periodically, we use the extreme learning machine to learn the law of speed changes from past data and predict the speed ranges that change over time to detect the data. In order to realize online data repair, a dual-window mechanism is proposed to transform the global optimal into the local optimal, and the traditional minimum change principle and median theorem are applied in the selection of the repair strategy. Aiming at the problem that the repair method based on the minimum change principle cannot correct consecutive abnormal points, through quantitative analysis, it is believed that the repair strategy should be the boundary of the repair candidate set. The experimental results obtained on the dataset show that the method proposed in this paper can get a better repair effect.

Design an Indexing Structure System Based on Apache Hadoop in Wireless Sensor Network

  • Keo, Kongkea;Chung, Yeongjee
    • Annual Conference of KIPS
    • /
    • 2013.05a
    • /
    • pp.45-48
    • /
    • 2013
  • In this paper, we proposed an Indexing Structure System (ISS) based on Apache Hadoop in Wireless Sensor Network (WSN). Nowadays sensors data continuously keep growing that need to control. Data constantly update in order to provide the newest information to users. While data keep growing, data retrieving and storing are face some challenges. So by using the ISS, we can maximize processing quality and minimize data retrieving time. In order to design ISS, Indexing Types have to be defined depend on each sensor type. After identifying, each sensor goes through the Indexing Structure Processing (ISP) in order to be indexed. After ISP, indexed data are streaming and storing in Hadoop Distributed File System (HDFS) across a number of separate machines. Indexed data are split and run by MapReduce tasks. Data are sorted and grouped depend on sensor data object categories. Thus, while users send the requests, all the queries will be filter from sensor data object and managing the task by MapReduce processing framework.

Data anomaly detection and Data fusion based on Incremental Principal Component Analysis in Fog Computing

  • Yu, Xue-Yong;Guo, Xin-Hui
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.10
    • /
    • pp.3989-4006
    • /
    • 2020
  • The intelligent agriculture monitoring is based on the perception and analysis of environmental data, which enables the monitoring of the production environment and the control of environmental regulation equipment. As the scale of the application continues to expand, a large amount of data will be generated from the perception layer and uploaded to the cloud service, which will bring challenges of insufficient bandwidth and processing capacity. A fog-based offline and real-time hybrid data analysis architecture was proposed in this paper, which combines offline and real-time analysis to enable real-time data processing on resource-constrained IoT devices. Furthermore, we propose a data process-ing algorithm based on the incremental principal component analysis, which can achieve data dimensionality reduction and update of principal components. We also introduce the concept of Squared Prediction Error (SPE) value and realize the abnormal detection of data through the combination of SPE value and data fusion algorithm. To ensure the accuracy and effectiveness of the algorithm, we design a regular-SPE hybrid model update strategy, which enables the principal component to be updated on demand when data anomalies are found. In addition, this strategy can significantly reduce resource consumption growth due to the data analysis architectures. Practical datasets-based simulations have confirmed that the proposed algorithm can perform data fusion and exception processing in real-time on resource-constrained devices; Our model update strategy can reduce the overall system resource consumption while ensuring the accuracy of the algorithm.

High-Volume Data Processing using Complex Event Processing Engine in the Web of Next Generation (차세대 웹 환경에서 Complex Event Processing 엔진을 이용한 대용량데이터 처리)

  • Kang, Man-Mo;Koo, Ra-Rok;Lee, Dong-Hyung
    • Journal of KIISE:Databases
    • /
    • v.37 no.6
    • /
    • pp.300-307
    • /
    • 2010
  • According to growth of web, data processing technology is developing. In the Web of next generation, high-speed or high-volume data processing technologies for various wire-wireless users, USN and RFID are developing too. In this paper, we propose a high-volume data processing technology using Complex Event Processing(CEP) engine. CEP is the technology to process complex events. CEP Engine is the following characteristics. First it collects a high-volume event(data). Secondly it analyses events. Finally it lets event connect to new actions. In other words, CEP engine collects, analyses, filters high-volume events. Also it extracts events using pattern-matching for registered events and new events. As the results extracted. We use it by an input event of other work, real-time response for demanded event and can trigger to database for only valid data.