Browse > Article
http://dx.doi.org/10.3745/KTCCS.2017.6.3.105

Design and Implementation of Initial OpenSHMEM Based on PCI Express  

Joo, Young-Woong (충북대학교 정보통신공학부)
Choi, Min (충북대학교 정보통신공학부)
Publication Information
KIPS Transactions on Computer and Communication Systems / v.6, no.3, 2017 , pp. 105-112 More about this Journal
Abstract
PCI Express is a bus technology that connects the processor and the peripheral I/O devices that widely used as an industry standard because it has the characteristics of high-speed, low power. In addition, PCI Express is system interconnect technology such as Ethernet and Infiniband used in high-performance computing and computer cluster. PGAS(partitioned global address space) programming model is often used to implement the one-sided RDMA(remote direct memory access) from multi-host systems, such as computer clusters. In this paper, we design and implement a OpenSHMEM API based on PCI Express maintaining the existing features of OpenSHMEM to implement RDMA based on PCI Express. We perform experiment with implemented OpenSHMEM API through a matrix multiplication example from system which PCs connected with NTB(non-transparent bridge) technology of PCI Express. The PCI Express interconnection network is currently very expensive and is not yet widely available to the general public. Nevertheless, we actually implemented and evaluated a PCI Express based interconnection network on the RDK evaluation board. In addition, we have implemented the OpenSHMEM software stack, which is of great interest recently.
Keywords
PGAS; OpenSHMEM; PCI Express; NTB; RDMA; One-Sided Communication;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Interconnect family statistics of supercomputer top 500 [Internet], https://www.top500.org/statistics/list. [Accessed: August, 28, 2016].
2 Y. W. Kim, Y. Ren, and W. Choi, "Design and Implementation of an Alternate System Interconnect based on PCI Express," Journal of the Institute of Electronics and Information Engineers, Vol.52, No.8, pp.74-85, Aug., 2015.   DOI
3 V. Krishnan, "Towards an Integrated IO and Clustering Solution using PCI Express," Cluster Computing, 2007 IEEE International Conference on, pp.259-266, Sept., 2007.
4 Jong Min Lee, Jung Hwa Lee, and Seong Woo Kim, "Implementation of a GPU Cluster System using Inexpensive Graphics Devices," Journal of Korea Multimedia Society, Vol.14, Issue 11, pp. 1458-1466, Nov., 2011.   DOI
5 J. Dinan, P. Balaji, Jeff R. Hammond, S. Krishnamoorthy, and V. Tipparaju, "Supporting the Global Arrays PGAS Model Using MPI One-Sided Communication," Parallel & Distributed Processing Symposium, 2012 IEEE 26th International, pp.739-750, May, 2012.
6 J. Breitbart, M. Schmidtobreick, and V. Heuveline, "Evaluation of the Global Address Space Programming interface(GASPI)," Parallel & Distributed Processing Symposium Workshops (IPDPSW), 2014 IEEE International, pp.717-726, May, 2014.
7 Hao Wang, Sreeram Potluri, Devendar Bureddy, Carlos Rosales, and Dhabaleswar K. Panda, "GPU-Aware MPI on RDMA-Enabled Clusters: Design, Implementation and Evaluation," IEEE Transactions on Parallel and Distributed Systems, pp.2595-2605, Oct., 2014.
8 Ryan E. Grant, Mohammad J. Rashti, Ahmad Afsahi, and Pavan Balaji, "RDMA Capable iWARP over Datagrams," Parallel & Distributed Processing Symposium (IPDPS), 2011 IEEE International, pp.628-639, May, 2011.
9 Weihang Jiang, Jiuxing Liu, hyun-Wook Jin, D.K. Panda, W. Gropp, and R. Thakur, "High performance MPI-2 one-sided communication over InfiniBand," Cluster Computing and the Grid, 2004. CCGrid 2004. IEEE International Symposium on, pp.531-538, Apr., 2004.
10 Yong-Hwan Lee, Do-Suk Kim and Sang Yoon Oh, "QoS and Flow Control Support on PCI Express Interface Architecture," Korea Institute of Information Technology Magazine, pp.45-52, Dec., 2009.
11 NTB white papers, AVAGO TECHNOLOGIES [Internet], http://www.avagotech.com/support/download-search. [Acce ssed: August, 29, 2016].
12 OpenSHMEM Specification document [Internet], http://open shmem.org/site/Specification. [Accessed: August, 29, 2016].
13 IXS600 PCI Express Gen 2 Switch, http://www.dolphinics.com.
14 Non-transparent Bridging with IDT 89HPES32NT24G2 PCI Express NTB Switch, Application Note AN-724, IDT.