Journal of KIISE:Computer Systems and Theory (한국정보과학회논문지:시스템및이론)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

A Parallel I/O System on Workstation Clustering Environment for Irregular Applications

비정형 응용을 위한 워크스테이션 클러스터링 환경에서의 병렬 입출력 시스템

  • Published : 2000.05.15
Download PDF
⟨ Previous Next ⟩

Abstract

Clusters of workstations (COW) are becoming an attractive option for parallel scientific computing, a field formerly reserved to the MPPs, because their cost-performance ratio is usuallybetter than that of comparable MPPS, and their hardware and software can be easily enhanced to thelatest generations. In this paper we present the design and implementation of our runtime library forclusters of workstations, called "Collective I/O Clustering". The library provides a friendlyprogramming model for the I/O of irregular applications on clusters of workstations, being completelyintegrated with the underlying communication and I/O system. In the collective I/O clustering, two I/Oconfigurations are possible. In the first I/O configuration, all processors allocated can act as I/Oservers as well as compute nodes. In the second I/O configuration, only a subset of processors canact as I/O servers, The compression and software caching facilities have been incorporated into thecollective 1/0 clustering to optimize the communication and I/O costs. All the performance results wereobtained on the IBM-SP machine, located at Argonne National Labs.

워크스테이션 클러스터 환경은 그 가격 대 성능비가 일반적으로 MPPS보다 좋고, 그 소프트웨어나 하드웨어가 쉽게 이후에 개선될 수 있기 때문에 병렬처리 분야에서 새로운 대안으로 연구되고 있다. 본 논문에서는 ‘집단적 입출력 클러스터링 (Collective I/O Clustering)’이라 불리는 워크스테이션 클러스터를 위한 실행사간 라이브러리의 설계 및 구현 방안을 제시한다. 이 라이브러리에서는 통신 및 입출력 시스템 하에서 완벽하게 통합되는 워크스테이션 클러스터 상에서 비정형 응용 프로그램의 입출력을 위해 ， 사용자에 친숙한 프로그래밍 모형을 제공한다,. 이 집단적 입출력 클러스터링에서는 두 가지 형태의 입출력 방식이 가능하다 첫 번째 입출력 방식에서 할당되는 모든 프로세서들은 연산 노드뿐만 아니라, 입출력 서버의 역할도 수행하는 형태이다. 두 번째 입출력 방식에서는 오직 일부분의 프로세서들만이 입출력 서버의 역할을 수행하는 형태이다. 그리고 본 논문에서는 통신과 입출력 비용을 최적화하기 위해 압축과 소프트웨어 캐슁 기능을 집단적 입출력 클러스터링에 적용한 결과를 보인다. 모든 성능실험 결과는 아르곤 연구소에서 보유하고 있는 IBM SP2를 사용하여 얻었다.

Keywords

References

  1. A.Arpaci-Dusseau and et.al. 'High-Performance Sorting on a Network of Workstations,' In SIGMOD'97 Proceedings, Tucson, Arizona, USA, May 1997 https://doi.org/10.1145/253260.253322
  2. P.Skordos. 'Parallel Simulation of Subsonic Fluid Dynamics on a Cluster of Workstations,' In Proceedings of the 4th IEEE Int. Symp. on High Performance Distributed Computing, Virginia, USA, Aug. 1995 https://doi.org/10.1109/HPDC.1995.518689
  3. T.Anderson, D.Culler, and D.Patterson. 'A Case for Network of Workstations: NOW,' IEEE Micro, Feb. 1995 https://doi.org/10.1109/40.342018
  4. J.T.Poole. 'Preliminary Survey of I/O Intensive Applications,' Technical Report CCSF-38, Scalable I/O Initiative, Caltech Concurrent Supercomputing Facilities, Caltech, 1994
  5. S.D.Sharma, R.Ponnusamy, B.Moon, Y.S.Hwang, R.Das, and J.Saltz. 'Run-time and Compile-time Support for Adaptive Irregular Problems,' In Supercomputing, IEEE Press, Nov. 1994 https://doi.org/10.1145/602770.602793
  6. L.A.Schoof and V.R.Yarberry. 'ExodusII: A Finite Element Data Model,' Technical Report SAND 94-2137, Sandia National Lab., Dec. 1994
  7. R.Thakur, E.Lusk, and W.Gropp. 'I/O Characterization of a Portable Astrophysics Application on the IBM SP and Intel Paragon,' Technical Report MCS-P534-0895, Argonne National Lab., Aug. 1995
  8. J.Carretero, J.No, S.Park, A.Choudhary, and P.Chen, 'Compassion: A Parallel I/O Runtime System Including Chucking and Compression for Irregular Applications,' In Proceedings of the International Conference on High-Performance Computing and Networking, Amsterdam, Holland, Apr. 1998
  9. J.No, S.S.Park, J.Carretero, A.Choudhary, and P.Chen, 'Design and Implementation of a Parallel I/O Runtime System for Irregular Applications,' In Proceedings of the 12th International Parallel Processing Symposium, Orlando, USA, Mar. 1998 https://doi.org/10.1109/IPPS.1998.669926
  10. R.Bordawekar, J.M.Rosario, and A.Choudhary. 'Design and Evaluation of Primitives for Parallel I/O,' In Proceedings of Supercomputing, pp.452-461, 1993 https://doi.org/10.1109/SUPERC.1993.1263493
  11. T.Mattson and G.Henry, 'The ASCI Option Red Supercomputer,' In Intel Supercomputer Users Group. Thirteenth Annual Conference, Albuquerque, USA, Jun. 1997
  12. T.H.Cormen and D.Kotz, 'Integrating Theory and Practice in Parallel File Systems,' Technical Report PCS-TR93-188, Dept. of MCS, Dartmouth College, Mar. 1993
  13. F.E.Bassow. Installing, Managing, and using the IBM AIX Parallel I/O File System, IBM Document No. SH34-6065-00, IBM Kingston, NY. Feb. 1995
  14. P.F.Corbett, S.J.Baylor, and D.G.Feitelson. 'Overview of the Vesta Parallel File System,' In Proceedings of '93 Workshop on Input/output in Parallel Computer Systems, pp.1-16, Dec. 1993
  15. Intel Corporation. Paragon User's Guide, Jun. 1994
  16. T.A.Welch. 'A Technique for High Performance Data Compression,' IEEE Computer, 17(6):8-19, Jun. 1984 https://doi.org/10.1109/MC.1984.1659158
  17. National Center for Supercomputing Applications. HDF reference manual version 4.1. Technical Report, University of Illinois, 1997
  18. K.E.Seamons and M.Winslett. 'A Data Management Approach for Handling Large Compressed Arrays in High Performance Computing,' In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computations, pp.119-128, Feb. 1995 https://doi.org/10.1109/FMPC.1995.380456
  19. R.Thakur, A.Choudhary, R.Bordawekar, S.More, and S.Kudatipidi, 'Passion: Optimized I/O for Parallel Systems,' IEEE Computer, Jun. 1996 https://doi.org/10.1109/MC.1996.10067
  20. R.Das, M.Uysal, J.Saltz, and Y.S.Hwang. 'Communication Optimizations for Irregular Scientific Computations on Distributed Memory Architectures,' Journal of Parallel and Distributed Computing, 22(3):462-479, Sep. 1994 https://doi.org/10.1006/jpdc.1994.1104