Leveraging Digital Cousins For Ensemble Q-learning In Large-scale Wireless Networks

Abstract

Optimizing large-scale wireless networks, including optimal resource management, power allocation, and throughput maximization, is inherently challenging due to their non-observable system dynamics and heterogeneous and complex nature. Herein, a novel ensemble Q-learning algorithm that addresses the performance and complexity challenges of the traditional Q-learning algorithm for optimizing wireless networks is presented. Ensemble learning with synthetic Markov Decision Processes is tailored to wireless networks via new models for approximating large state-space observable wireless networks. In particular, digital cousins are proposed as an extension of the traditional digital twin concept wherein multiple Q-learning algorithms on multiple synthetic Markovian environments are run in parallel and their outputs are fused into a single Q-function. Convergence analyses of key statistics and Q-functions and derivations of upper bounds on the estimation bias and variance are provided. Numeri

Stats

• citations7
• S2 citations—
• github stars0
• HF likes0
• heat score6.77
• arxiv keybozkus2024leveraging

Related papers

• Coverage Analysis Of Multi-environment Q-learning Algorithms For Wireless Network Optimization (2024)0.00
• A Multi-agent Multi-environment Mixed Q-learning For Partially Decentralized Wireless Network Optimization (2024)0.00
• Implications Of Decentralized Q-learning Resource Allocation In Wireless Networks (2017)0.00
• Multi-timescale Ensemble Q-learning For Markov Decision Process Policy Optimization (2024)6.34
• Factorized Q-learning For Large-scale Multi-agent Systems (2018)11.58
• Deep Reinforcement Learning For Distributed And Uncoordinated Cognitive Radios Resource Allocation (2022)0.00
• Provable Performance Bounds For Digital Twin-driven Deep Reinforcement Learning In Wireless Networks: A Novel Digital-twin Bisimulation Metric (2025)0.00
• Offline Reinforcement Learning For Wireless Network Optimization With Mixture Datasets (2023)9.59