• International journal of advanced smart convergence
• /
• 제10권4호
• /
• pp.278-288
• /
• 2021

Recently, IoT systems are cloud-based, so that continuous and large amounts of data collected from sensor nodes are processed in the data server through the cloud. However, in the centralized configuration of large-scale cloud computing, computational processing must be performed at a physical location where data collection and processing take place, and the need for edge computers to reduce the network load of the cloud system is gradually expanding. In this paper, a cluster system consisting of 6 inexpensive Raspberry Pi boards was constructed to perform fast data processing. And we propose "Kubernetes cluster system(KCS)" for processing large data collection and analysis by model distribution and data pipeline method. To compare the performance of this study, an ensemble model of deep learning was built, and the accuracy, processing performance, and processing time through the proposed KCS system and model distribution were compared and analyzed. As a result, the ensemble model was excellent in accuracy, but the KCS implemented as a data pipeline proved to be superior in processing speed..

• Journal of Astronomy and Space Sciences
• /
• 제20권4호
• /
• pp.403-416
• /
• 2003

There are common features, in both imaging surveys and image processing, between astronomical observations and remote sensing. Handling large amounts of data, in an easy and fast way, has become a common issue. Implementing pipeline software can be a solution to the problem, one which allows the processing of various kinds of data automatically. As a case study, the development of pipeline software for the EIS (European Southern Observatory Imaging Survey) is introduced. The EIS team has been conducting a sky survey to provide candidate targets to the 250 VLTs (Very Large Telescopes) observations. The survey data have been processed in a sequence of five major data corrections and reductions, i.e. preprocessing, flat fielding, photometric and astrometric corrections, source extraction, and coaddition. The processed data are eventually distributed to the users. In order to provide automatic processing of the vast volume of observed data, pipeline software has been developed. Because of the complexity of objects and different characteristic of each process, it was necessary to analyze the whole works of the EIS survey program. The overall tasks of the EIS are identified, and the scheme of the EIS pipeline software is defined. The system structure and the processes are presented, and in-depth flow charts are analyzed. During the analyses, it was revealed that handling the data flow and managing the database are important for the data processing. These analyses may also be applied to many other fields which require image processing.

• 대한전자공학회:학술대회논문집
• /
• 대한전자공학회 2002년도 ITC-CSCC -3
• /
• pp.1823-1826
• /
• 2002

On modern computer culture, the high quality data is required in multimedia systems. So, the technology of data compression fur data transmission is necessary now. This paper presents the pipeline architecture for the low and column address generator of 2D DCT/IDCT (Discrete Cosine Transform/Inverse Discrete Cosine Transform. In the proposed architecture, the area of hardware is reduced by using the DA (distributed arithmetic) method and applies the concepts of pipeline to the parallel architecture. As a result the designed pipeline of the low and column address generator for 2D DCT/IDCT architecture is implemented with an efficiency and high speed compared with the non-pipeline architecture.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권4호
• /
• pp.1146-1165
• /
• 2022

A huge amount of data in the form of videos and images is being produced owning to advancements in sensor technology. Use of low performance commodity hardware coupled with resource heavy image processing and analyzing approaches to infer and extract actionable insights from this data poses a bottleneck for timely decision making. Current approach of GPU assisted and cloud-based architecture video analysis techniques give significant performance gain, but its usage is constrained by financial considerations and extremely complex architecture level details. In this paper we propose a data pipeline system that uses open-source tools such as Apache Spark, Kafka and OpenCV running over commodity hardware for video stream processing and image processing in a distributed environment. Experimental results show that our proposed approach eliminates the need of GPU based hardware and cloud computing infrastructure to achieve efficient video steam processing for face detection with increased throughput, scalability and better performance.

• 한국멀티미디어학회:학술대회논문집
• /
• 한국멀티미디어학회 2003년도 춘계학술발표대회논문집
• /
• pp.14-18
• /
• 2003

This paper presents the pipeline architecture for the low and column address generator of 2D DCT/IDCT(Discrete Cosine Transform/Inverse Discrete Cosine Transform). For the real time process of image data, it is required that high speed operation and small size hardware In the proposed architecture, the area of hardware is reduced by using the DA(distributed arithmetic) method and applying the concepts of pipeline on the parallel architecture. As a results, the designed pipeline of the low and column address generator for 2D DCT/IDCT architecture is implemented with an efficiency and high speed compared as the non-pipeline architecture. And the operation speed is improved about 50％ up. The design for the proposed pipeline architecture of DCT/IDCT is coded using VHDL.

• 한국정보처리학회:학술대회논문집
• /
• 한국정보처리학회 2009년도 추계학술발표대회
• /
• pp.121-122
• /
• 2009

인터넷 상에서 정보량이 급격히 증가함에 따라 ISP들은 데이터를 효과적으로 처리하고 분석하기 위한 방법을 연구하고 있다. 대표적으로 Google에서는 대용량의 분산 데이터 처리 기법인 MapReduce 모델을 개발하였다. 본 논문에서는 기존 MapReduce 모델에 Pipeline 방식을 적용하여 성능을 개선한 Pipeline-MapReduce 기법을 제안한다. 그리고 실험을 통해 제안 기법이 기존 기법에 비해 빠른 처리 결과를 나타냄을 보여준다.

• Journal of Platform Technology
• /
• 제11권4호
• /
• pp.71-79
• /
• 2023

이 논문에서는 기계학습 워크로드의 특징을 분석하고 이를 기반으로 기계학습 워크로드의 성능 향상을 위한 분산 인-메모리 캐싱 기법을 제안한다. 기계학습 워크로드의 핵심은 모델 학습이며 모델 학습은 컴퓨팅 집약적 (Computation Intensive)인 작업이다. 쿠버네티스 기반 클라우드 환경에서 컴퓨팅 프레임워크와 스토리지를 분리한 구조에서 기계학습 워크로드를 수행하는 것은 자원을 효과적으로 할당할 수 있지만, 네트워크 통신을 통해 IO가 수행되야 하므로 지연이 발생할 수 있다. 이 논문에서는 이런 환경에서 수행되는 머신러닝 워크로드의 성능을 향상하기 위한 분산 인-메모리 캐싱 기법을 제안한다. 특히, 제안하는 방법은 쿠버네티스 기반의 머신러닝 파이프라인 관리 도구인 쿠브플로우를 고려하여 머신러닝 워크로드에 필요한 데이터를 분산 인-메모리 캐시에 미리 로드하는 새로운 방법을 제안한다.

• 대한전자공학회논문지
• /
• 제26권8호
• /
• pp.1217-1226
• /
• 1989

This paper describes a design of hardware system for real time automatic diagnosis of ECG arrhythmia based on pipeline processor consisting of three microcomputer. ECG data is acquisited by 12 bit A/D converter with hardware QRS triggered detector. Four diagnostic parameters-heart rate, morpholigy, axis, and ST segment-are used for the classification and the diagnosis of arrhythmia. The functions of the main CPU were distributed and processed with three microcomputers. Therefore the effective data process and the real time process using microcomputer can be obtained. The interconnection structure consisting of two common memory unit is designed to decrease the delay time caused by data transfer between processors and be which the delay time can be taken 1% of one clock period.

• 대한의용생체공학회:의공학회지
• /
• 제9권1호
• /
• pp.79-86
• /
• 1988

This paper describes the design of a hardware systenl for ECG feature extraction based on pipeline processor consistinsf of three microcomputers. ECG data is acquisited by 12 bit A/D converter with hardware QRS triggered detector. Four diagnostic parameters parameters-heart rate, morPhology, axis, and 57 segment-are used for the classification and the diagnosis of arrhythmia. The functions of the main CPU were distributed and processed with three microcomputers. Therefore the effective data process and the real time process using microcomputer can be obtained. The interconnection structure consisting of two common memory units is designed to decrease the delay time caused by data transfer between processors and designed by which the delay time can be taken Loye of one clock period.

• 대한전기학회:학술대회논문집
• /
• 대한전기학회 1988년도 전기.전자공학 학술대회 논문집
• /
• pp.731-733
• /
• 1988

This paper describes a design of hardware system for real time automatic diagnosis of ECG arrhythmia based on pipeline processor consisting of the three microcomputer. ECG data is acquisited by 12 bit A/D converter with hardware QRS triggered detector. Four diagnostic parameters - heart rate, morphology, axis, and ST segment - are used for the classification and the diagnosis of arrhythmia. The functions of the main CPU were distributed and processed with three microcomputers. There-fore the effective data process and the real time process using microcomputer can be obtained. The interconnection structure consisting of two common memory units is designed to decrease the delay time caused by data transfer between processors and by which the delay time can be taken 1 % of one clock period.