Offline Reinforcement Learning For Wireless Network Optimization With Mixture Datasets

Abstract

The recent development of reinforcement learning (RL) has boosted the adoption of online RL for wireless radio resource management (RRM). However, online RL algorithms require direct interactions with the environment, which may be undesirable given the potential performance loss due to the unavoidable exploration in RL. In this work, we first investigate the use of *offline* RL algorithms in solving the RRM problem. We evaluate several state-of-the-art offline RL algorithms, including behavior constrained Q-learning (BCQ), conservative Q-learning (CQL), and implicit Q-learning (IQL), for a specific RRM problem that aims at maximizing a linear combination \{of sum and\} 5-percentile rates via user scheduling. We observe that the performance of offline RL for the RRM problem depends critically on the behavior policy used for data collection, and further propose a novel offline RL solution that leverages heterogeneous datasets collected by different behavior policies. We show that with a

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keyyang2023offline

Related papers

• Offline And Distributional Reinforcement Learning For Radio Resource Management (2024)0.00
• Average Reward Reinforcement Learning For Wireless Radio Resource Management (2025)2.26
• Offline Reinforcement Learning With Imbalanced Datasets (2023)0.00
• Resource Management In Wireless Networks Via Multi-agent Deep Reinforcement Learning (2020)16.43
• Reinforcement Learning For Individual Optimal Policy From Heterogeneous Data (2025)0.00
• Deep Reinforcement Learning For Distributed And Uncoordinated Cognitive Radios Resource Allocation (2022)0.00
• Deployment-efficient Reinforcement Learning Via Model-based Offline Optimization (2020)0.00
• Implications Of Decentralized Q-learning Resource Allocation In Wireless Networks (2017)0.00