A Multi-agent Multi-environment Mixed Q-learning For Partially Decentralized Wireless Network Optimization

Abstract

Q-learning is a powerful tool for network control and policy optimization in wireless networks, but it struggles with large state spaces. Recent advancements, like multi-environment mixed Q-learning (MEMQ), improves performance and reduces complexity by integrating multiple Q-learning algorithms across multiple related environments so-called digital cousins. However, MEMQ is designed for centralized single-agent networks and is not suitable for decentralized or multi-agent networks. To address this challenge, we propose a novel multi-agent MEMQ algorithm for partially decentralized wireless networks with multiple mobile transmitters (TXs) and base stations (BSs), where TXs do not have access to each other's states and actions. In uncoordinated states, TXs act independently to minimize their individual costs. In coordinated states, TXs use a Bayesian approach to estimate the joint state based on local observations and share limited information with leader TX to minimize joint cost. The

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybozkus2024a

Related papers

• Coverage Analysis Of Multi-environment Q-learning Algorithms For Wireless Network Optimization (2024)0.00
• Leveraging Digital Cousins For Ensemble Q-learning In Large-scale Wireless Networks (2024)6.77
• Implications Of Decentralized Q-learning Resource Allocation In Wireless Networks (2017)0.00
• MFC-EQ: Mean-field Control With Envelope Q-learning For Moving Decentralized Agents In Formation (2024)0.00
• Mitigating Relative Over-generalization In Multi-agent Reinforcement Learning (2024)0.00
• Multi-agent Determinantal Q-learning (2020)0.00
• Multi-timescale Ensemble Q-learning For Markov Decision Process Policy Optimization (2024)6.34
• Distributed Q-learning With State Tracking For Multi-agent Networked Control (2020)0.00