CXL Memory as Persistent Memory for Disaggregated HPC: A Practical Approach

Yehonatan Fridman, Suprasad Mutalik Desai, Navneet Singh, Thomas Willhalm, Gal Oren

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In the landscape of High-Performance Computing (HPC), the quest for efficient and scalable memory solutions remains paramount. The advent of Compute Express Link (CXL) introduces a promising avenue with its potential to function as a Persistent Memory (PMem) solution in the context of disaggregated HPC systems. This paper presents a comprehensive exploration of CXL memory's viability as a candidate for PMem, supported by physical experiments conducted on cutting-edge multi-NUMA nodes equipped with CXL-attached memory prototypes. Our study not only benchmarks the performance of CXL memory but also illustrates the seamless transition from traditional PMem programming models to CXL, reinforcing its practicality. To substantiate our claims, we establish a tangible CXL prototype using an FPGA card embodying CXL 1.1/2.0 compliant endpoint designs (Intel FPGA CXL IP). Performance evaluations, executed through the STREAM and STREAM-PMem benchmarks, showcase CXL memory's ability to mirror PMem characteristics in App-Direct and Memory Mode while achieving impressive bandwidth metrics with Intel 4th generation Xeon (Sapphire Rapids) processors. The results elucidate the feasibility of CXL memory as a persistent memory solution, outperforming previously established benchmarks. In contrast to published DCPMM results, our CXL-DDR4 memory module offers comparable bandwidth to local DDR4 memory configurations, albeit with a moderate decrease in performance. The modified STREAM-PMem application underscores the ease of transitioning programming models from PMem to CXL, thus underscoring the practicality of adopting CXL memory. The sources of this work are available at: https://github.com/Scientific-Computing-Lab-NRCN/STREAMer.

Original languageEnglish
Title of host publicationProceedings of 2023 SC Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023
PublisherAssociation for Computing Machinery
Pages983-994
Number of pages12
ISBN (Electronic)9798400707858
DOIs
StatePublished - 12 Nov 2023
Externally publishedYes
Event2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023 - Denver, United States
Duration: 12 Nov 202317 Nov 2023

Publication series

NameACM International Conference Proceeding Series

Conference

Conference2023 International Conference on High Performance Computing, Network, Storage, and Analysis, SC Workshops 2023
Country/TerritoryUnited States
CityDenver
Period12/11/2317/11/23

Bibliographical note

Publisher Copyright:
© 2023 ACM.

Keywords

  • CXL
  • HPC
  • Intel Optane DCPMM
  • Memory disaggregation
  • Persistent Memory (PMem)
  • STREAM
  • STREAM-PMem
  • STREAMer

Fingerprint

Dive into the research topics of 'CXL Memory as Persistent Memory for Disaggregated HPC: A Practical Approach'. Together they form a unique fingerprint.

Cite this