Reinforcement Learning For Datacenter Congestion Control
2021 · Chen Tessler, Yuval Shpigelman, Gal Dalal, et al.
Abstract
We approach the task of network congestion control in datacenters using Reinforcement Learning (RL). Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, no such learning-based algorithms have shown practical potential in this domain. Evidently, the most popular recent deployments rely on rule-based heuristics that are tested on a predetermined set of benchmarks. Consequently, these heuristics do not generalize well to newly-seen scenarios. Contrarily, we devise an RL-based algorithm with the aim of generalizing to different configurations of real-world datacenter networks. We overcome challenges such as partial-observability, non-stationarity, and multi-objectiveness. We further propose a policy gradient algorithm that leverages the analytical structure of the reward function to approximate its derivative and improve stability. We show that this scheme outperforms alternative popular RL approaches, and generalizes to sc
Authors
(none)
Tags
Stats
Related papers
- Offline Reinforcement Learning For Wireless Network Optimization With Mixture Datasets (2023)9.59
- Generalization In Reinforcement Learning For Radio Access Networks (2025)0.00
- Communication-efficient Policy Gradient Methods For Distributed Reinforcement Learning (2018)13.05
- Intervention-assisted Policy Gradient Methods For Online Stochastic Queuing Network Optimization: Technical Report (2024)0.00
- Reinforcement Learning For Intensity Control: An Application To Choice-based Network Revenue Management (2024)0.00
- Quantifying The Impact Of Non-stationarity In Reinforcement Learning-based Traffic Signal Control (2020)8.35
- Online Reinforcement Learning In Non-stationary Context-driven Environments (2023)0.00
- Unified Algorithms For RL With Decision-estimation Coefficients: PAC, Reward-free, Preference-based Learning, And Beyond (2022)5.24