• Title/Summary/Keyword: Parallel disk I/O

Search Result 29, Processing Time 0.023 seconds

A Concurrency/Coherency Control Approach using the I/O node for the Shared Disk Parallel Database (공유 디스크 병렬 데이타베이스에서 입출력 노드를 이용한 동시성/응집성 제어 기법)

  • 김용걸;김양우;진성일;임기욱
    • The Journal of Information Technology and Database
    • /
    • v.3 no.2
    • /
    • pp.25-38
    • /
    • 1996
  • 병렬 데이타베이스 소프트웨어 구조 중 공유 디스크 구조는 트랜잭션간의 병렬도 향상, 적재 균형 용이, 데이타 재할당 용이 등의 장점을 가지고 있어 병렬 데이타베이스 구조 중 가장 효율적인 성능이 기대되고 있다. 그러나 공유 디스크 구조는 동시성/응집성 제어를 위한 추가적인 메세지의 증가로 네트워크 트래픽이 증가되는 문제를 가지고 있으며 이러한 문제를 완화시키고자 하는 연구가 계속되고 있다. 본 논문에서는 공유 디스크 구조의 동시성/응집성 제어를 위한 추가적인 메세지를 감소시키는 기법을 제안하고, 기존 기법과의 비교를 위해 성능 모델을 제시하였으며, 이를 통한 시뮬레이션을 수행하여 성능을 분석하였다.

  • PDF

Optimizing Fsync Performance with Dynamic Queue Depth Adaptation

  • Park, Daejun;Kim, Min Ji;Shin, Dongkun
    • JSTS:Journal of Semiconductor Technology and Science
    • /
    • v.15 no.5
    • /
    • pp.570-576
    • /
    • 2015
  • Existing flash storage devices such as universal flash storage and solid state disk support command queuing to improve storage I/O bandwidth. Command queuing allows multiple read/write requests to be pending in a device queue. Because multi-channel and multi-way architecture of flash storage devices can handle multiple requests simultaneously, command queuing is an indispensable technique for utilizing parallel architecture. However, command queuing can be harmful to the latency of fsync system call, which is critical to application responsiveness. We propose a dynamic queue depth adaptation technique, which reduces the queue depth if user application is expected to send fsync calls. Experiments show that the proposed technique reduces the fsync latency by 79% on average compared to the original scheme.

WADPM : Workload-Aware Dynamic Page-level Mapping Scheme for SSD based on NAND Flash Memory (낸드 플래시 메모리 기반 SSD를 위한 작업부하 적응형 동적 페이지 매핑 기법)

  • Ha, Byung-Min;Cho, Hyun-Jin;Eom, Young-Ik
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.37 no.4
    • /
    • pp.215-225
    • /
    • 2010
  • The NAND flash memory based SSDs are considered to replace the existing HDDs. To maximize the I/O performance, SSD is composed of several NAND flash memories in parallel. However, to adopt the hybrid mapping scheme in SSD may cause degradation of the I/O performance. In this paper, we propose a new mapping scheme for the SSD called WADPM. WADPM loads only necessary mapping information into RAM and dynamically adjusts the size of mapping information in the RAM. So, WADPM avoids the shortcoming of page-level mapping scheme that requires too large mapping table. Performance evaluation using simulations shows that I/O performance of WADPM is 3.5 times better than the hybrid-mapping scheme and maximum size of mapping table of WADPM is about 50% in comparison with the page-level mapping scheme.

Performance Evaluation of VBR MPEG Video Storage and Retrieval Schemes in a VOD System (VOD 시스템에서의 가변 비트율 MPEG 비디오 저장 및 검색 기법의 성능 평가)

  • 전용희;박정숙
    • Journal of Korea Multimedia Society
    • /
    • v.4 no.1
    • /
    • pp.13-28
    • /
    • 2001
  • In a VOD(Vide-On-Demand) system, video data are generally stored in magnetic disk array. In order to provide real-time requirement for data retrieval, video streams must be delivered continuously to the clients such that the delivery of continuous media can be guaranteed in a timely fashion. Compared to the increased performance of processors and networks, the performance of magnetic disk systems have improved only modestly. In order to improve the performance of storage system, disk array system is proposed and used. The array system improves I/O performance by placing disks in parallel and retrieving data concurrently. In this paper, two approaches are considered in order to access the video data in a VOD system, which are CTL(Constant Time Length) and CDL(Constant Data Length) access policies. Disk scheduling policies are also classified into the two categories and compared in terms of the maximum allowable video streams with different degrees of disk array synchronization, under the mixed environments in which both data access policy and disk scheduling policy are considered. Among the compared scheduling policies, LOOK was shown to have the best performance. In terms of degree of disk synchronization, more gain was achieved with large degree of synchronization. In comparisons of performance of CTL and CDL, CTL was proved to have a little superior performance in terms of number of maximum allowable streams.

  • PDF

Measuring Hadoop Optimality by Lorenz Curve (로렌츠 커브를 이용한 하둡 플랫폼의 최적화 지수)

  • Kim, Woo-Cheol;Baek, Changryong
    • The Korean Journal of Applied Statistics
    • /
    • v.27 no.2
    • /
    • pp.249-261
    • /
    • 2014
  • Ever increasing "Big data" can only be effectively processed by parallel computing. Parallel computing refers to a high performance computational method that achieves effectiveness by dividing a big query into smaller subtasks and aggregating results from subtasks to provide an output. However, it is well-known that parallel computing does not achieve scalability which means that performance is improved linearly by adding more computers because it requires a very careful assignment of tasks to each node and collecting results in a timely manner. Hadoop is one of the most successful platforms to attain scalability. In this paper, we propose a measurement for Hadoop optimization by utilizing a Lorenz curve which is a proxy for the inequality of hardware resources. Our proposed index takes into account the intrinsic overhead of Hadoop systems such as CPU, disk I/O and network. Therefore, it also indicates that a given Hadoop can be improved explicitly and in what capacity. Our proposed method is illustrated with experimental data and substantiated by Monte Carlo simulations.

A Scheme on High-Performance Caching and High-Capacity File Transmission for Cloud Storage Optimization (클라우드 스토리지 최적화를 위한 고속 캐싱 및 대용량 파일 전송 기법)

  • Kim, Tae-Hun;Kim, Jung-Han;Eom, Young-Ik
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.37 no.8C
    • /
    • pp.670-679
    • /
    • 2012
  • The recent dissemination of cloud computing makes the amount of data storage to be increased and the cost of storing the data grow rapidly. Accordingly, data and service requests from users also increases the load on the cloud storage. There have been many works that tries to provide low-cost and high-performance schemes on distributed file systems. However, most of them have some weaknesses on performing parallel and random data accesses as well as data accesses of frequent small workloads. Recently, improving the performance of distributed file system based on caching technology is getting much attention. In this paper, we propose a CHPC(Cloud storage High-Performance Caching) framework, providing parallel caching, distributed caching, and proxy caching in distributed file systems. This study compares the proposed framework with existing cloud systems in regard to the reduction of the server's disk I/O, prevention of the server-side bottleneck, deduplication of the page caches in each client, and improvement of overall IOPS. As a results, we show some optimization possibilities on the cloud storage systems based on some evaluations and comparisons with other conventional methods.

Performance Analysis on Declustering High-Dimensional Data by GRID Partitioning (그리드 분할에 의한 다차원 데이터 디클러스터링 성능 분석)

  • Kim, Hak-Cheol;Kim, Tae-Wan;Li, Ki-Joune
    • The KIPS Transactions:PartD
    • /
    • v.11D no.5
    • /
    • pp.1011-1020
    • /
    • 2004
  • A lot of work has been done to improve the I/O performance of such a system that store and manage a massive amount of data by distributing them across multiple disks and access them in parallel. Most of the previous work has focused on an efficient mapping from a grid ceil, which is determined bY the interval number of each dimension, to a disk number on the assumption that each dimension is split into disjoint intervals such that entire data space is GRID-like partitioned. However, they have ignored the effects of a GRID partitioning scheme on declustering performance. In this paper, we enhance the performance of mapping function based declustering algorithms by applying a good GRID par-titioning method. For this, we propose an estimation model to count the number of grid cells intersected by a range query and apply a GRID partitioning scheme which minimizes query result size among the possible schemes. While it is common to do binary partition for high-dimensional data, we choose less number of dimensions than needed for binary partition and split several times along that dimensions so that we can reduce the number of grid cells touched by a query. Several experimental results show that the proposed estimation model gives accuracy within 0.5% error ratio regardless of query size and dimension. We can also improve the performance of declustering algorithm based on mapping function, called Kronecker Sequence, which has been known to be the best among the mapping functions for high-dimensional data, up to 23 times by applying an efficient GRID partitioning scheme.

Efficient Method to Support Mobile Virtualization-based Cloud Resource Management (모바일 가상화기반 클라우드 자원관리를 지원하는 효율적 방법)

  • Kang, Yongho;Jang, Changbok;Lee, Wanjik;Heo, Seokyeol;Kim, Jooman
    • Journal of Digital Convergence
    • /
    • v.12 no.2
    • /
    • pp.277-283
    • /
    • 2014
  • Recently, various cloud service has been being provided on mobile devices as well as desktop pc and server computer. Also, Smartphone users are very rapidly increasing, and they are using it for enjoying various services(cloud service, game, banking service, mobile office, etc.). So, research to utilize resources on mobile device has been conducted. In this paper, We have suggested efficient method of cloud resource management by using information of available physical resources(CPU, memory, storage, etc.) between mobile devices, and information of physical resource in mobile device. Suggested technology is possible to guarantee real-time process and efficiently manage resources.

Speed-up Techniques for High-Resolution Grid Data Processing in the Early Warning System for Agrometeorological Disaster (농업기상재해 조기경보시스템에서의 고해상도 격자형 자료의 처리 속도 향상 기법)

  • Park, J.H.;Shin, Y.S.;Kim, S.K.;Kang, W.S.;Han, Y.K.;Kim, J.H.;Kim, D.J.;Kim, S.O.;Shim, K.M.;Park, E.W.
    • Korean Journal of Agricultural and Forest Meteorology
    • /
    • v.19 no.3
    • /
    • pp.153-163
    • /
    • 2017
  • The objective of this study is to enhance the model's speed of estimating weather variables (e.g., minimum/maximum temperature, sunshine hour, PRISM (Parameter-elevation Regression on Independent Slopes Model) based precipitation), which are applied to the Agrometeorological Early Warning System (http://www.agmet.kr). The current process of weather estimation is operated on high-performance multi-core CPUs that have 8 physical cores and 16 logical threads. Nonetheless, the server is not even dedicated to the handling of a single county, indicating that very high overhead is involved in calculating the 10 counties of the Seomjin River Basin. In order to reduce such overhead, several cache and parallelization techniques were used to measure the performance and to check the applicability. Results are as follows: (1) for simple calculations such as Growing Degree Days accumulation, the time required for Input and Output (I/O) is significantly greater than that for calculation, suggesting the need of a technique which reduces disk I/O bottlenecks; (2) when there are many I/O, it is advantageous to distribute them on several servers. However, each server must have a cache for input data so that it does not compete for the same resource; and (3) GPU-based parallel processing method is most suitable for models such as PRISM with large computation loads.