• Title/Summary/Keyword: High-speed Data Processing

Search Result 756, Processing Time 0.031 seconds

MAHA-FS : A Distributed File System for High Performance Metadata Processing and Random IO (MAHA-FS : 고성능 메타데이터 처리 및 랜덤 입출력을 위한 분산 파일 시스템)

  • Kim, Young Chang;Kim, Dong Oh;Kim, Hong Yeon;Kim, Young Kyun;Choi, Wan
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.2
    • /
    • pp.91-96
    • /
    • 2013
  • The application field of supercomputing systems are changing to support into the field for both a large-volume data processing and high-performance computing at the same time such as bio-applications. These applications require high-performance distributed file system for storage management and efficient high-speed processing of large amounts of data that occurs. In this paper, we introduce MAHA-FS for supercomputing systems for processing large amounts of data and high-performance computing, providing excellent metadata operation performance and IO performance. It is shown through performance analysis that MAHA-FS provides excellent performance in terms of the metadata processing and random IO processing.

Developemet of noncontact velocity tracking algorithm for 3-dimensional high speed flows using digital image processing technique (디지털 화상처리를 이용한 유동장의 비접촉 3차원 고속류 계측법의 개발)

  • 도덕희
    • Journal of Advanced Marine Engineering and Technology
    • /
    • v.23 no.2
    • /
    • pp.259-269
    • /
    • 1999
  • A new algorithm for measuring 3-D velocity components of high speed flows were developed using a digital image processing technique. The measuring system consists of three CCD cameras an optical instrument called AOM a digital image grabber and a host computer. The images of mov-ing particles arranged spatially on a rotation plate are taken by two or three CCD cameras and are recorderd onto the image grabber or a video tape recoder. The three-dimensionl velocity com-ponents of the particles are automatically obtained by the developed algorithm In order to verify the validity of this technique three-dimensional velocity data sets obtained from a computer simu-lation of a backward facing step flow were used as test data for the algorithm. an uncertainty analysis associated with the present algorithm is systematically evaluated, The present technique is proved to be used as a tookl for the measurement of unsteady three-dimensional fluid flows.

  • PDF

The Design and Implementation of Frequency Domain Sampling Surface Acoustic Wave Sensor Platform (Frequency Domain Sampling 방식의 Surface Acoustic Wave Sensor Platform 설계 및 구현)

  • Joh, Yool-Hee;Kim, Young-Kil
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.05a
    • /
    • pp.220-223
    • /
    • 2012
  • Generally, SAW device, which uses Time Domain Sampling, requires high speed AD converter because SAW device (TDS) needs high sampling speed as much as its high data speed. However, the high price of AD converter discourages makers from using it. On the other hand, SAW device, which uses Frequency Domain Sampling, does not required high speed AD converter because SAW device (FDS) does not need high sampling speed. It is very efficient in price comparison to its performance because high processing speed of SAW device (FDS) can be implemented using low price Embedded Systems. The purpose of the thesis is to solve the issues above by designing and realizing SAW device (FDS) using SAW sensor for TDS.

  • PDF

The Design and Implementation of Frequency Domain Sampling Surface Acoustic Wave Sensor Platform using Cortex-A8 (Cortex-A8을 이용한 Frequency Domain Sampling 방식의 Surface Acoustic Wave Sensor Platform 설계 및 구현)

  • Joh, Yool-hee;Kim, Young-kil
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.10a
    • /
    • pp.312-315
    • /
    • 2012
  • Generally, SAW device, which uses Time Domain Sampling, requires high speed AD converter because SAW device (TDS) needs high sampling speed as much as its high data speed. However, the high price of AD converter discourages makers from using it. On the other hand, SAW device, which uses Frequency Domain Sampling, does not required high speed AD converter because SAW device (FDS) does not need high sampling speed. It is very efficient in price comparison to its performance because high processing speed of SAW device (FDS) can be implemented using low price Embedded Systems. The purpose of the thesis is to solve the issues above by designing and realizing SAW device (FDS) using SAW sensor for TDS.

  • PDF

Improvement of Image Sensor Performance through Implementation of JPEG2000 H/W for Optimal DWT Decomposition Level

  • Lee, Choel;Kim, BeomSu;Jeon, ByungKook
    • International journal of advanced smart convergence
    • /
    • v.6 no.1
    • /
    • pp.68-75
    • /
    • 2017
  • In this paper, a particular application of digital photos, remote sensing, remote shooting air moving, high-resolution and high compression of medical images required by remote shooting of JPEG2000 standard applied in the field of hardware design, production was implemented. JPEG2000 standard for image compression using the software implementation of the processing speed is very slow compared to conventional JPEG disadvantages, and also the standard of JPEG2000 DWT (Discrete wavelet transform) to improve the level of compression for image data if processing speed is a phenomenon that has degraded. In order to solve these JPEG2000 compression / decompression groups were designed and applied. In this paper, the optimal JPEG2000 compression / reservoir hardware by changing the level for still image compression, faster computation speed and quality has shown improvement.

High-Volume Data Processing using Complex Event Processing Engine in the Web of Next Generation (차세대 웹 환경에서 Complex Event Processing 엔진을 이용한 대용량데이터 처리)

  • Kang, Man-Mo;Koo, Ra-Rok;Lee, Dong-Hyung
    • Journal of KIISE:Databases
    • /
    • v.37 no.6
    • /
    • pp.300-307
    • /
    • 2010
  • According to growth of web, data processing technology is developing. In the Web of next generation, high-speed or high-volume data processing technologies for various wire-wireless users, USN and RFID are developing too. In this paper, we propose a high-volume data processing technology using Complex Event Processing(CEP) engine. CEP is the technology to process complex events. CEP Engine is the following characteristics. First it collects a high-volume event(data). Secondly it analyses events. Finally it lets event connect to new actions. In other words, CEP engine collects, analyses, filters high-volume events. Also it extracts events using pattern-matching for registered events and new events. As the results extracted. We use it by an input event of other work, real-time response for demanded event and can trigger to database for only valid data.

DEVELOPMENT OF REAL-TIME DATA REDUCTION PIPELINE FOR KMTNet (KMTNet 실시간 자료처리 파이프라인 개발)

  • Kim, D.J.;Lee, C.U.;Kim, S.L.;Park, B.G.
    • Publications of The Korean Astronomical Society
    • /
    • v.28 no.1
    • /
    • pp.1-6
    • /
    • 2013
  • Real-time data reduction pipeline for the Korea Microlensing Telescope Network (KMTNet) was developed by Korea Astronomy and Space Science Institute (KASI). The main goal of the data reduction pipeline is to find variable objects and to record their light variation from the large amount of observation data of about 200 GB per night per site. To achieve the goal we adopt three strategic implementations: precision pointing of telescope using the cross correlation correction for target fields, realtime data transferring using kernel-level file handling and high speed network, and segment data processing architecture using the Sun-Grid engine. We tested performance of the pipeline using simulated data which represent the similar circumstance to CTIO (Cerro Tololo Inter-American Observatory), and we have found that it takes about eight hours for whole processing of one-night data. Therefore we conclude that the pipeline works without problem in real-time if the network speed is high enough, e.g., as high as in CTIO.

The Conceptual Design of Mass Memory Unit for High Speed Data Processing in the STSAT-3 (고속 데이터 처리를 위한 과학기술위성 3호 대용량 메모리 유닛의 개념 설계)

  • Seo, In-Ho;Oh, Dae-Soo;Myung, Noh-Hoon
    • Journal of the Korean Society for Aeronautical & Space Sciences
    • /
    • v.38 no.4
    • /
    • pp.389-394
    • /
    • 2010
  • This paper describes the conceptual design of mass memory unit for high speed data processing and mass memory management in the STSAT-3 compared to that of STSAT-2. The FPGA directly controls the data receiving from two payloads with the maximum 100Mbps speed and 32Gb mass memory management to satisfy these requirements. We used SRAM-based FPGA from XILINX having fast operating speed and large logic cells. Therefore, the Triple Modular Redundancy(TMR) and configuration memory scrubbing techniques will also be used to protect FPGA from Single Event Upset(SEU) in space.

Implementation of High Speed Big Data Processing System using In Memory Data Grid in Semiconductor Process (반도체 공정에서 인 메모리 데이터 그리드를 이용한 고속의 빅데이터 처리 시스템 구현)

  • Park, Jong-Beom;Lee, Alex;Kim, Tony
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.15 no.5
    • /
    • pp.125-133
    • /
    • 2016
  • Data processing capacity and speed are rapidly increasing due to the development of hardware and software in recent time. As a result, data usage is geometrically increasing and the amount of data which computers have to process has already exceeded five-thousand transaction per second. That is, the importance of Big Data is due to its 'real-time' and this makes it possible to analyze all the data in order to obtain accurate data at right time under any circumstances. Moreover, there are many researches about this as construction of smart factory with the application of Big Data is expected to have reduction in development, production, and quality management cost. In this paper, system using In-Memory Data Grid for high speed processing is implemented in semiconductor process which numerous data occur and improved performance is proven with experiments. Implemented system is expected to be possible to apply on not only the semiconductor but also any fields using Big Data and further researches will be made for possible application on other fields.

Relation and Variation Trend between the Behavior of the Pantograph vs. the Vehicle Running Speed in Korean High Speed Train (한국형 고속전철용 판토그라프의 거동 특성과 열차속도와의 상관관계와 경향)

  • 목진용;박춘수;조용현;최강윤;김기환
    • Proceedings of the Korean Society for Noise and Vibration Engineering Conference
    • /
    • 2003.11a
    • /
    • pp.170-176
    • /
    • 2003
  • The pantograph for Korean High Speed Train was developed and had been evaluating by through "G7- R&D for home grown high speed train technology". In this study, a relation in mechanical aspect between the train running speed and the current collecting performance of the pantograph is conducted.'for this study, a measuring system for current collecting performance and mechanical characteristics is developed and installed on the prototype Korean High Speed Train, and measurement is conducted while the train runs on the test track. The measuring system is composed of video monitoring system and telemetry & data processing unit. It monitors whether the hazard behavior in the pantograph is occurs or not, and measures acceleration and vertical contact force between the pan head and catenary. Through this study, evaluation of a mechanical vibration characteristics and trend of the pantograph and a interface performance of pantograph - catenary up to 200㎞/h train speed are facilitated.

  • PDF