Fedformer: Contextual Federation With Attention In Reinforcement Learning

Abstract

A core issue in multi-agent federated reinforcement learning is defining how to aggregate insights from multiple agents. This is commonly done by taking the average of each participating agent's model weights into one common model (FedAvg). We instead propose FedFormer, a novel federation strategy that utilizes Transformer Attention to contextually aggregate embeddings from models originating from different learner agents. In so doing, we attentively weigh the contributions of other agents with respect to the current agent's environment and learned relationships, thus providing a more effective and efficient federation. We evaluate our methods on the Meta-World environment and find that our approach yields significant improvements over FedAvg and non-federated Soft Actor-Critic single-agent methods. Our results compared to Soft Actor-Critic show that FedFormer achieves higher episodic return while still abiding by the privacy constraints of federated learning. Finally, we also demonstr

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhebert2022fedformer

Related papers

• Dearfsac: An Approach To Optimizing Unreliable Federated Learning Via Deep Reinforcement Learning (2022)0.00
• Momentum For The Win: Collaborative Federated Reinforcement Learning Across Heterogeneous Environments (2024)0.00
• Federated Distributional Reinforcement Learning With Distributional Critic Regularization (2026)0.00
• On The Linear Speedup Of Personalized Federated Reinforcement Learning With Shared Representations (2024)0.00
• A Multi-agent Reinforcement Learning Approach For Efficient Client Selection In Federated Learning (2022)15.00
• Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis (2025)0.00
• Fedhpd: Heterogeneous Federated Reinforcement Learning Via Policy Distillation (2025)2.26
• Multi-task Federated Reinforcement Learning With Adversaries (2021)0.00