Causal Influence Detection For Improving Efficiency In Reinforcement Learning

Abstract

Many reinforcement learning (RL) environments consist of independent entities that interact sparsely. In such environments, RL agents have only limited influence over other entities in any particular situation. Our idea in this work is that learning can be efficiently guided by knowing when and what the agent can influence with its actions. To achieve this, we introduce a measure of *situation-dependent causal influence* based on conditional mutual information and show that it can reliably detect states of influence. We then propose several ways to integrate this measure into RL algorithms to improve exploration and off-policy learning. All modified algorithms show strong increases in data efficiency on robotic manipulation tasks.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyseitzer2021causal

Related papers

• Social Influence As Intrinsic Motivation For Multi-agent Deep Reinforcement Learning (2018)0.00
• Situation-dependent Causal Influence-based Cooperative Multi-agent Reinforcement Learning (2023)5.24
• Reducing Action Space For Deep Reinforcement Learning Via Causal Effect Estimation (2025)0.00
• A Roadmap Towards Improving Multi-agent Reinforcement Learning With Causal Discovery And Inference (2025)0.00
• Learning By Doing: An Online Causal Reinforcement Learning Framework With Causal-aware Policy (2024)1.56
• Causal Reinforcement Learning Using Observational And Interventional Data (2021)0.00
• Learning Nonlinear Causal Reductions To Explain Reinforcement Learning Policies (2025)0.00
• Discovering Causality For Efficient Cooperation In Multi-agent Environments (2023)0.00