Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs

Benjamin Fuhrer, Yuval Shpigelman, Chen Tessler, Shie Mannor, Gal Chechik, Eitan Zahavi, Gal Dalal

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

As communication protocols evolve, datacenter network utilization increases. As a result, congestion is more frequent, causing higher latency and packet loss. Combined with the increasing complexity of workloads, manual design of congestion control (CC) algorithms becomes extremely difficult. This calls for the development of AI approaches to replace the human effort. Unfortunately, it is currently not possible to deploy AI models on network devices due to their limited computational capabilities. Here, we offer a solution to this problem by building a computationally-light solution based on a recent reinforcement learning CC algorithm [1, RL-CC]. We reduce the inference time of RL-CC by x500 by distilling its complex neural network into decision trees. This transformation enables real-time inference within the μ-sec decision-time requirement, with a negligible effect on quality. We deploy the transformed policy on NVIDIA NICs in a live cluster. Compared to popular CC algorithms used in production, RL-CC is the only method that performs well on all benchmarks tested over a large range of number of flows. It balances multiple metrics simultaneously: bandwidth, latency, and packet drops. These results suggest that data-driven methods for CC are feasible, challenging the prior belief that handcrafted heuristics are necessary to achieve optimal performance.

Original languageEnglish
Title of host publicationProceedings - 23rd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2023
EditorsYogesh Simmhan, Ilkay Altintas, Ana-Lucia Varbanescu, Pavan Balaji, Abhinandan S. Prasad, Lorenzo Carnevale
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages331-343
Number of pages13
ISBN (Electronic)9798350301199
DOIs
StatePublished - 2023
Event23rd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2023 - Bangalore, India
Duration: 1 May 20234 May 2023

Publication series

NameProceedings - 23rd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2023

Conference

Conference23rd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2023
Country/TerritoryIndia
CityBangalore
Period1/05/234/05/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

  • RDMA
  • congestion control
  • datacenter networks
  • distillation
  • gradient boosting trees
  • reinforcement learning

Fingerprint

Dive into the research topics of 'Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs'. Together they form a unique fingerprint.

Cite this