Browse > Article
http://dx.doi.org/10.4218/etrij.2021-0335

Distributed memory access architecture and control for fully disaggregated datacenter network  

Kyeong-Eun Han (Network Research Division, Electronics and Telecommunications Research Institute)
Ji Wook Youn (Network Research Division, Electronics and Telecommunications Research Institute)
Jongtae Song (Network Research Division, Electronics and Telecommunications Research Institute)
Dae-Ub Kim (Network Research Division, Electronics and Telecommunications Research Institute)
Joon Ki Lee (Network Research Division, Electronics and Telecommunications Research Institute)
Publication Information
ETRI Journal / v.44, no.6, 2022 , pp. 1020-1033 More about this Journal
Abstract
In this paper, we propose novel disaggregated memory module (dMM) architecture and memory access control schemes to solve the collision and contention problems of memory disaggregation, reducing the average memory access time to less than 1 ㎲. In the schemes, the distributed scheduler in each dMM determines the order of memory read/write access based on delay-sensitive priority requests in the disaggregated memory access frame (dMAF). We used the memory-intensive first (MIF) algorithm and priority-based MIF (p-MIF) algorithm that prioritize delay-sensitive and/or memory-intensive (MI) traffic over CPU-intensive (CI) traffic. We evaluated the performance of the proposed schemes through simulation using OPNET and hardware implementation. Our results showed that when the offered load was below 0.7 and the payload of dMAF was 256 bytes, the average round trip time (RTT) was the lowest, ~0.676 ㎲. The dMM scheduling algorithms, MIF and p-MIF, achieved delay less than 1 ㎲ for all MI traffic with less than 10% of transmission overhead.
Keywords
disaggregated datacenter; disaggregated memory access; resource-centric;
Citations & Related Records
연도 인용수 순위
  • Reference
1 P. X. Gao, A. Narayan, S. Karandikar, J. Carreira, S. Han, R. Agarwal, S. Ratnasamy, and S. Shenker, Network requirements for resource disaggregation, (OSDI'16: Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation, Berkeley, CA, USA), 2016, pp. 249-264.
2 Y. Cheng, R. Lin, M. De Andrade, L. Wosinska, and J. Chen, Disaggregated data centers: Challenges and tradeoffs, IEEE Commun. Mag. 58 (2019), no. 2, 20-26.
3 Y. Cheng, M. De Andrade, L. Wosinska, and J. Chen, Resource disaggregation versus integrated servers in data center: Impact of internal transmission capacity limitation, (European Conference on Optical Communication, Rome, Italy), Sept. 2018. https://doi.org/10.1109/ECOC.2018.8535214
4 S. Han, N. Egi, A. Panda, S. Ratnasamy, G. Shi, and S. Shenker, Network support for resource disaggregation in nextgeneration datacenters, (HotNets-XII: Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks, New York, NY, USA), Nov. 2013. https://doi.org/10.1145/2535771.2535778   DOI
5 S. Di, D. Kondo, and F. Cappello, Characterizing cloud applications on a Google Data Center, (42nd International Conference on Parallel Processing, Lyon, France), Oct. 2013. https://doi. org/10.1109/ICPP.2013.56   DOI
6 G. Zervas, H. Yuan, A. Saljoghei, Q. Chen, and V. Mishra, Optically disaggregated data centers with minimal remote memory latency: Technologies, architectures, and resource allocation [invited], J. Opt. Commun. Netw. 10 (2018), no. 2, A270-A285.   DOI
7 A. Klimovic, C. Kozyrakis, E. Thereska, B. John, and S. Kumar, Flash storage disaggregation, (EuroSys '16: Proceedings of the Eleventh European Conference on Computer Systems, New York, NY, USA), Apr. 2016. https://doi.org/10.1145/ 2901318.2901337   DOI
8 M. K. Aguilera, N. Amit, I. Calciu, X. Deguillard, J. Gandhi, P. Subrahmanyam, L. Suresh, K. Tati, R. Venkatasubramanian, and M. Wei, Remote memory in the age of fast network, (SoCC '17: Proceedings of the 2017 Symposium on Cloud Computing, Santa Clara, CA, USA), Sept. 2017. https://doi.org/10.1145/ 3127479.3131612   DOI
9 D. Buragohain, A. Ghogare, T. Patel, M. Vutukuru, and P. Kulkarni, DiME: A performance emulator for disaggregated memory architectures, (APSys '17: Proceedings of the 8th AsiaPacific Workshop on Systems, New York, NY, USA), Sept. 2017. https://doi.org/10.1145/3124680.3124731   DOI
10 C. Li, H. Franke, C. Parris, B. Abali, M. Kesavan, and V. Chang, Composable architecture for rack scale big data computing, Future Gener. Comput. Syst. 67 (2016), 180-193.   DOI
11 K. Shi, S. Lange, I. Haller, D. Cletheroe, R. Behrendt, B. Thomsen, F. Karinou, K. Jozwik, P. Costa, and H. Ballani, System demonstration of nanosecond wavelength switching with burst-mode PAM4 transceiver, (45th European Conference on Optical Communication, Dublin, Ireland), Sept. 2019. https://doi.org/10.1049/cp.2019.1034   DOI
12 J. Weiss, R. Dangel, J. Hofrichter, F. Horst, D. Jubin, N. Meier, A. La Porta, and B. J. Offrein, Optical interconnects for disaggregated resources in future datacenters, (The European Conference on Optical Communication, Cannes, France), Sept. 2014. https://doi.org/10.1109/ECOC.2014.6964255   DOI
13 K. Clark, H. Ballani, P. Bayvel, D. Cletheroe, T. Gerard, I. Haller, K. Jozwik, K. Shi, B. Thomsen, P. Watts, and H. Williams, Sub-nanosecond clock and data recovery in an optically-switched data centre network, (European Conference on Optical Communication, Rome, Italy), Sept. 2018. https://doi. org/10.1109/ECOC.2018.8535333   DOI
14 X. Guo, F. Yan, X. Xue, G. Exarchakos, and N. Calabretta, Performance assessment of a novel rack-scale disaggregated data center with fast optical switch, (Optical Fiber Communications Conference and Exhibition, Sand Diego, CA, USA), Mar. 2019.
15 CCIX Consortium, https://www.ccixconsortium.com, (2019).
16 J. Youn, K. E. Han, H. Lee, J. Song, and J. Lee, Experimental demonstration of optically disaggregated memory for edge data centers, (Opto-Electronics and Communications Conference, Taipei, Taiwan), Oct. 2020. https://doi.org/10.1109/ OECC48412.2020.9273657   DOI
17 N. Terzenidis, M. Moralis-Pegios, G. Mourgias-Alexandris, T. Alexoudi, K. Vyrsokinos, and N. Pleros, High-port and lowlatency optical switches for disaggregated data centers: The Hipoλaos switch architecture, J. Opt. Comm. Netw. 10 (2018), no. 7, B102-B116.   DOI
18 D. Syrivelis, A. Reale, M. Gazzetti, and P. Hofstee, Dynamic synthesis of disaggregated hardware platforms via cache coherent interconnect optical bridge, (European Conference on Optical Communication, Dublin, Ireland), Sept. 2019, pp. 1-3.
19 M. Bielski, I. Syrigos, K. Katrinis, D. Syrivelis, A. Reale, D. Theodoropoulos, N. Alachiotis, D. Pnevmatikatos, E. H. Pap, G. Zervas, and V. Mishra, dReDBox: Materializing a full-stack rack-scale system prototype of a next-generation disaggregated datacenter, (Design, Automation & Test in Europe Conference & Exhibition, Dresden, Germany), Mar. 2018. https://doi.org/10.23919/DATE.2018.8342174   DOI
20 G. Zervas, F. Jiang, Q. Chen, V. Mishra, H. Yuan, K. Katrinis, D. Syrivelis, A. Reale, D. Pnevmatikatos, M. Enrico, and N. Parsons, Disaggregated compute, memory and network systems: A new era for optical data centre architectures, (Optical Fiber Communications Conference and Exhibition, Los Angeles, CA, USA), Mar. 2017.
21 N. Terzenidis, M. Moralis-Pegios, G. Mourgias-Alexandris, K. Vyrsokinos, and N. Pleros, High-port low-latency optical switch architecture with optical feed-forward buffering for 256-node disaggregated data centers, Opt. Express 26 (2018), no. 7, 8756-8766.   DOI
22 F. Yan, W. Miao, O. Raz, and N. Calabretta, OPSquare: A flat DCN architecture based on flow-controlled optical packet switches, Opt. Commun. Netw. 9 (2019), no. 4, 291-303.
23 S. McCreary and K. C. Claffy, Trends in wide area IP traffic patterns, Cooperative Association for Internet Data Analysis, USA. Available from: http://www.caida.org [last accessed 2017].