[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.7236/IJASC.2020.9.3.253

Implementation of Light-weight I/O Stack for NVMe-over-Fabrics

Ahn, Sungyong (School of Computer Science and Engineering, Pusan National University)

Publication Information

International journal of advanced smart convergence / v.9, no.3, 2020 , pp. 253-259 More about this Journal

Abstract

Most of today's large-scale cloud systems and enterprise data centers are distributing resources to improve scalability and resource utilization. NVMe-over-Fabric protocol allows submitting NVMe commands to a remote NVMe SSD through RDMA (Remote Direct Memory Access) network. It is attracting attention recently because it is possible to construct a disaggregation storage system with low latency through the protocol. However, the current I/O stack of NVMe-over-Fabric has an inefficient structure for maintaining compatibility with the traditional I/O stack. Therefore, in this paper, we propose a new mechanism to reduce I/O latency and CPU overhead by modifying I/O path of NVMe-over-Fabric to pass through legacy block layer. According to the performance evaluation results, the proposed mechanism is able to reduce the I/O latency and CPU overhead by up to 22% and 24% compared to the existing NVMe-over-Fabrics protocol, respectively.

Keywords

Large-scale Cloud System; NVMe-over-Fabric; Storage Area Network; RDMA; NVMe SSDs;

Citations & Related Records

Reference

1	A. Klimovic, C. Kozyrakis, E. Thereska, B. John, and S. Kumar, "Flash storage disaggregation," in Proc. 11th European Conference on Computer Systems (EuroSys '16), pp. 1-15, April 18-21, 2016. DOI: https://doi.org/10.1145/2901318.2901337
2	NVM Express Base Specification Revision 1.4a, https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4a- 2020.03.09-Ratified.pdf.
3	NVMe_Over_Fabrics, https://nvmexpress.org/wp-content/uploads/NVMe_Over_Fabrics.pdf
4	Flexible I/O Tester, https://github.com/axboe/fio.
5	R. Mittal, A. Shpiner, A. Panda, E. Zahavi, A. Krishnamurthy, S. Ratnasamy, and S. Shenker, "Revisiting network support for RDMA," in Proc. the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM '18), pp. 313-326, Aug. 20-25, 2018 DOI: https://doi.org/10.1145/3230543.3230557
6	Z. Guz, H. Li, A. Shayesteh, and V. Balakrishnan, "NVMe-over-fabrics performance characterization and the path to low-overhead flash disaggregation," in Proc. 10th ACM International Systems and Storage Conference, pp. 1-9, May 22-24, 2017. DOI: https://doi.org/10.1145/3078468.3078483
7	D. Han and B. Nam, "Improving Access to HDFS using NVMeoF," in Proc. 2019 IEEE International Conference on Cluster Computing (CLUSTER), pp. 1-2, Sep. 23-26, 2019. DOI: https://doi.org/10.1109/CLUSTER.2019.8890996
8	J. Choi, H. Eom, and H. Yeom, "An Evaluation of Shared-Disk File Systems Using NVMe-over-Fabrics," in Proc. Korea Software Congress 2017, pp. 85-87, Dec. 20-22, 2017.
9	J. Choi, H. Yeom, and H. Han, “An Evaluation of NVMe-over-Fabrics on PCIe JBOF SSDs,” Journal of KIISE, Vol. 46, No. 6, pp. 499-505, June 2019. DOI: https://doi.org/10.5626/JOK.2019.46.6.499 DOI
10	Y. Zhu, W. Yu, B. Jiao, K. Mohror, A. Moody and F. Chowdhury, "Efficient User-Level Storage Disaggregation for Deep Learning," in Proc. 2019 IEEE International Conference on Cluster Computing (CLUSTER), pp. 1-12, Sept. 23-26, 2019. DOI: https://doi.org/10.1109/CLUSTER.2019.8891023
11	NVM Express over Fabrics Specification Revision 1.1, https://nvmexpress.org/wp-content/uploads/NVMe-over- Fabrics-1.1-2019.10.22-Ratified.pdf.