ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Gen-Z memory pool system implementation and performance measurement

  • Kwon, Won-ok (Department of Future Computing Research Division, Electronics and Telecommunications Research Institute) ;
  • Sok, Song-Woo (Department of Future Computing Research Division, Electronics and Telecommunications Research Institute) ;
  • Park, Chan-ho (Department of Future Computing Research Division, Electronics and Telecommunications Research Institute) ;
  • Oh, Myeong-Hoon (Department of Computer Engineering, Honam University) ;
  • Hong, Seokbin (Department of Computer Software, University of Science and Technology)
  • Received : 2021.03.09
  • Accepted : 2021.07.01
  • Published : 2022.06.10
Download PDF
⟨ Previous Next ⟩

Abstract

The Gen-Z protocol is a memory semantic protocol between the memory and CPU used in computer architectures with large memory pools. This study presents the implementation of the Gen-Z hardware system configured using Gen-Z specification 1.0 and reports its performance. A hardware prototype of a DDR4 Gen-Z memory pool with an optimized character, a block device driver, and a file system for the Gen-Z hardware was designed. The Gen-Z IP was targeted to the FPGA, and a 512 GB Gen-Z memory pool was configured on an ×86 server. In the experiments, the latency and throughput of the Gen-Z memory were measured and compared with those of the local memory, SATA SSD, and NVMe using character or block device interfaces. The Gen-Z hardware exhibited superior throughput and latency performance compared with SATA SSD and NVMe at block sizes under 4 kB. The MySQL and File IO benchmark of Gen-Z showed good write performance in all block sizes and threads. Besides, it showed low latency in RocksDB's fillseq dbbench using the ext4 direct access filesystem.

Keywords

Acknowledgement

The authors would like to acknowledge the Gen-Z consortium and HPE for their technical assistance in the preparation of this study. This work was supported by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) (no. 2018-0-00503, Researches on next-generation memory-centric computing system architecture).

References

  1. D. Reinsel, J. Gantz, and J. Rydning, Data age 2025: The evolution of data to life-critical don't focus on big data; focus on the data that's big, IDC, Apr. 2017.
  2. P. Chaudhary, Data centric computing, in Proc. SPXXL/-SCICOMP Summer 2011, (Edinburgh, Scotland), May 2009.
  3. A. Boroumand et al., Google workloads for consumer devices: Mitigating data movement bottlenecks, in Proc. Int. Conf. Archit. Support Programm. Lang. Oper. Syst. (ASPLOS) (Williamsburg, VA, USA), Mar. 2018, pp. 316-331.
  4. O. Mutlu, Memory-centric computing in the big data era, FMS Special Session Invited Talk, ETH Zurich, Aug. 8, 2019.
  5. R. Balasubramonian et al., Near-data processing: Insights from a micro-46 workshop, IEEE Micro 34 (2014), no. 4, 36-42. https://doi.org/10.1109/mm.2014.55
  6. Y. Park, IBM data centric systems & OpenPOWER, HPC User Forum, May 6, 2017.
  7. H. Kwonet al., Signal integrity analysis of system interconnection module of high-density server supporting serial RapidIO, ETRI J. 41 (2019), no. 5, 670-683. https://doi.org/10.4218/etrij.2018-0021
  8. H. Kim, C. G. Lyuh, and Y. Kwon, Automated optimization for memory-efficient high-performance deep neural network accelerators, ETRI J. 42 (2020), no. 4, 505-517. https://doi.org/10.4218/etrij.2020-0125
  9. K. Keeton, The machine: An architecture for memory-centric computing, in Proc. Workshop Runtim Oper. Syst. Supercomput. (ROSS), (Portland, OR, USA), June 2015.
  10. I. Calciu et al., Project pberry: FPGA acceleration for remote memory, in Proc. Workshop Hot Top. Oper. Syst. (Bertinoro, Italy), May 2019.
  11. Gen-Z Specifications, The Gen-Z Consortium, Available from: https://genzconsortium.org/ [retrieved Oct. 2020].
  12. CCIX Specifications, CCXI Consortium, Available from: https://www.ccixconsortium.com/ [retrieved Aug. 2020].
  13. OpenCAPI Specifications, OpenCAPI Consortium, Available from: https://opencapi.org/ [retrieved June 2020].
  14. CXL Consortium. Available from: https://www.computeexpresslink.org/ [retrieved Aug. 2020].
  15. S. Hong, W. Kwon, and M. H. Oh, Hardware implementation and analysis of Gen-Z protocol for memory-centric architecture, IEEE Access 8 (2020), 127244-127253. https://doi.org/10.1109/access.2020.3008227
  16. PMDK, Persistent memory programming, Available from: https://pmem.io/pmdk/ [retrieved Aug. 2020].
  17. Gen-Z core specification, ver.1.0, 2018.
  18. P. Knebel et al., Gen-Z chipsetfor exascale fabrics, in Proc. IEEE Hot chips 31 Symp. (Cupertino, CA, USA), Aug. 2019.
  19. IntelliProp Gen-Z IP core, IntelliProp. Available from: https://www.intelliprop.com/ [retrieved Aug. 2020].
  20. Gen-Z Memory Module (ZMM) Smart Modular Technology, Available from: https://www.smartm.com/ [retrieved Aug. 2019].
  21. T. Morgan, Gen-Z Memory Servers Loom on the Horizon, The Next Platform, Jan. 9, 2020, Available from: https://www.nextplatform.com/2020/01/09/gen-z-memory-servers-loom-on-the-horizon/
  22. XUPP3R, Xilinx UltraScale+ 3/4-length PCIe board User Guide, BittWare, Sept. 2019.
  23. Using the Memmap kernel option, persistent memory documentation, Available from: https://docs.pmem.io/persistent-memory/getting-started-guide/creating-development-environments/linux-environments/linux-memmap [retrieved Aug. 2020].
  24. NVMe "disk" bandwidth and latency for batched block requests, Available from: https://panthema.net/2019/0322-nvme-batched-block-access-speed/ [retrieved Mar. 2019].
  25. J. Izraelevitz et al., Basic performance measurements of the intel optane DC persistent memory module, arXiv preprint, CoRR, 2019, arXiv: 1903.05714.