Goal Discovery With Causal Capacity For Efficient Reinforcement Learning

Abstract

Causal inference is crucial for humans to explore the world, which can be modeled to enable an agent to efficiently explore the environment in reinforcement learning. Existing research indicates that establishing the causality between action and state transition will enhance an agent to reason how a policy affects its future trajectory, thereby promoting directed exploration. However, it is challenging to measure the causality due to its intractability in the vast state-action space of complex scenarios. In this paper, we propose a novel Goal Discovery with Causal Capacity (GDCC) framework for efficient environment exploration. Specifically, we first derive a measurement of causality in state space, *i.e.,* causal capacity, which represents the highest influence of an agent's behavior on future trajectories. After that, we present a Monte Carlo based method to identify critical points in discrete state space and further optimize this method for continuous high-dimensional environments.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyu2025goal

Related papers

• Boosting Efficiency In Task-agnostic Exploration Through Causal Knowledge (2024)0.00
• Learning By Doing: An Online Causal Reinforcement Learning Framework With Causal-aware Policy (2024)1.56
• Reducing Action Space For Deep Reinforcement Learning Via Causal Effect Estimation (2025)0.00
• Learning Causal Overhypotheses Through Exploration In Children And Computational Models (2022)0.00
• A Roadmap Towards Improving Multi-agent Reinforcement Learning With Causal Discovery And Inference (2025)0.00
• Disentangling Causal Effects For Hierarchical Reinforcement Learning (2020)0.00
• From Kicking To Causality: Simulating Infant Agency Detection With A Robust Intrinsic Reward (2025)0.00
• Causal Influence Detection For Improving Efficiency In Reinforcement Learning (2021)0.00