Browse > Article
http://dx.doi.org/10.5573/ieie.2015.52.8.074

Design and Implementation of an Alternate System Interconnect based on PCI Express  

Kim, Young Woo (Cloud Computing Research Department, ETRI)
Ren, Ye (Cloud Computing Research Department, ETRI)
Choi, WonHyuk (Cloud Computing Research Department, ETRI)
Publication Information
Journal of the Institute of Electronics and Information Engineers / v.52, no.8, 2015 , pp. 74-85 More about this Journal
Abstract
PCI Express is a well-known and widely used de-facto system bus standard for connecting among a processor and IO devices. PCI Express is originated from old PCI standard, and its most of applications are limited to be used within a PC or server system. But, because of its fast speed, low power consumption, and good protocol efficiency, it is considered as one of a good candidate for an alternate system interconnect for many years. In this paper, we present design, implementation and early evaluation of an alternate system interconnect by utilizing PCI Express. The developed alternate system interconnect using PCI Express (named PCIeLINK) utilizes non-transparent bridging (NTB) technic which generally used in fail-over system in PCI and PCI Express. By using NTB technic, PCI Express device can be extended to outside of a system without electrical and logical problems arising during system boot and enumeration. To build up an alternate system interconnect, we designed and implemented a network interface card having multiple PCI Express ${\times}4$ connections (theoretically 20 Gbps) and tested, The early test results revealed that an ${\times}4$ port in the card showed 8.6 Gbps peak performance for bulk transmission and 5.1 Gbps peak for normal TCP/IP transfer.
Keywords
PCI Express; Interconnect; Non Transparent; PCIeLINK;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Young Woo Kim, et. al., "Technology and Trends of High Performance Processors," Electronics and telecommunications trends, vol.25, no.5, pp.123-136, 2010.
2 Interconnect Family Statistics, TOP500. org accessed May 14, 2015, http://www.top500.org/statistics/list/
3 "The 2015 Ethernet Roadmap," white paper from Ethernet alliance accessed May 13, 2015, http://www.ethernetalliance.org/roadmap/
4 "InfiniBand Roadmap," InfiniBand Trade Association accessed May13, 2015, http://www.infinibandta.org/content/pages.php?pg=technology_overview
5 "Annual Meeting of Members Presentation," PCI-SIG accessed May13, 2015, https://www.pcisig.com/members/downloads/PCISIG_Annual_Meeting_of_Members_2013_Final.pdf
6 "Why FPU Generator?" Floating-Point Unit Generator accessed May15, 2015, https://sites.google.com/a/stanford.edu/fpgen/why
7 Wonok Kwon, et. al., "PCI Express Gen3 System Design using High-speed Signal Integrity Analysis," Journal of the Institute of Electronics and Information Engineers, vol.52, no.4, pp.125-132, 2015.   DOI   ScienceOn
8 Wonok Kwon, et. al., "ALTERA Embedded Gigabit Transceiver Measurement for PCI Express Protocol," Journal of the Institute of Electronics and Information Engineers, vol.41, no.4, pp.359-367, 2004.
9 Vijay Medury, "PCI Express in Clustering," High Speed Interconnects Seminar, Linley Group, Nov., 2010.
10 JohnB yrne, et. al., "Power-Efficient Networking for Balanced System Designs: Early Experiences with PCIe," HotPower '11 Proceedings of the 4th Workshop on Power-Aware Computing and Systems, Article No. 3, 2011.
11 Ahmed Bu-Khamsin, "Socket Direct Protocol over PCI Express Interconnect: Design, Implementation and Evaluation," MS Thesis, Simon Fraser University, 2012.
12 T. Hanawa,T. Boku, S. Miura, T. Okamoto, M. Sato, et al., "Low-Power and High-Performance Communication Mechanism for Dependable Embedded Systems,"International Workshop on Innovative Architecture for Future Generation High-Performance Processors and Systems, pp. 67-73, 2008.
13 T. Hanawa, Y. Kodama, T. Boku, M. Sato, "Interconnection Network for Tightly Coupled Accelerators Architecture," IEEE 21st Annual Symposium on High-Performance Interconnects, San Jose, USA, pp. 79-82, Aug. 2013.
14 Young Woo Kim, et. al., "Design of MAHA Supercomputing System for Human Genome Analysis," KIPS transactions on software and dataengineering, vol.2, no.2, pp.81-90, 2013.   DOI   ScienceOn
15 Young Woo Kim, "MAHA-Manycore HPC System for Bio-Applications," The Second KIISE-KOCSEA HPC SIG Joint Workshop, Denver, USA, Nov., 2013.
16 D. Percival, "PCI Express Clustering," PCI-SIG Developers Conference, Israel, 2011.
17 Young Woo Kim, et. al., "Implementation of System Interconnection Device Using PCI Express," Proceedings on 2014 IEIE Summer Conference, pp.515-518, Jeju Island, Korea, 2014.
18 Ye Ren, et. al., "A Preliminary Implementation of PCIeLINK and Its Performance Evaluation," Proceedings on 2014 IEIE Summer Conference, pp.519-522, Jeju Island, Korea, 2014.
19 Ye Ren, et. al., "Implementation of System Interconnection Devices Using PCI Express," IEEE International Conference on Consumer Electronics, Las Vegas, USA, pp.300-301, Jan. 2015.
20 IDT's PCIe Gen2 Switch family Non-Trans patent Operation, Application Note, IDT, 2009
21 "SFF-8436 Specification for QSFP+ 10 Gbs 4X PLUGGABLE TRANSCEIVER" SFF Committee accessed Apr. 13, 2012, ftp://ftp.seagate.com/sff/SFF-8436.PDF
22 $IDT^{(R)}$ 89HPES24NT6AG2 PCI $Express^{(R)}$ Switch User Manual, IDT accessed May 25, 2012.
23 WonHyuk Choi, et.al., "A Design of Dedicated Communication module for PCI Express network device," KIISE Proceedings on Korea Computer Congress 2015, pp 95-97, Jeju Island, Korea, 2015.
24 NetPIPE, http://bitspjoule.org/netpipe/
25 Netperf, http://www.netperf.org/netperf/