CIER: A Novel Experience Replay Approach With Causal Inference In Deep Reinforcement Learning

Abstract

In the training process of Deep Reinforcement Learning (DRL), agents require repetitive interactions with the environment. With an increase in training volume and model complexity, it is still a challenging problem to enhance data utilization and explainability of DRL training. This paper addresses these challenges by focusing on the temporal correlations within the time dimension of time series. We propose a novel approach to segment multivariate time series into meaningful subsequences and represent the time series based on these subsequences. Furthermore, the subsequences are employed for causal inference to identify fundamental causal factors that significantly impact training outcomes. We design a module to provide feedback on the causality during DRL training. Several experiments demonstrate the feasibility of our approach in common environments, confirming its ability to enhance the effectiveness of DRL training and impart a certain level of explainability to the training proces

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2024cier

Related papers

• A Roadmap Towards Improving Multi-agent Reinforcement Learning With Causal Discovery And Inference (2025)0.00
• Reccover: Detecting Causal Confusion For Explainable Reinforcement Learning (2022)0.00
• Learning By Doing: An Online Causal Reinforcement Learning Framework With Causal-aware Policy (2024)1.56
• Causal-paced Deep Reinforcement Learning (2025)0.00
• Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms And Applications (2025)0.00
• D3HRL: A Distributed Hierarchical Reinforcement Learning Approach Based On Causal Discovery And Spurious Correlation Detection (2025)0.00
• Using Contrastive Samples For Identifying And Leveraging Possible Causal Relationships In Reinforcement Learning (2022)0.00
• Causal Reinforcement Learning Using Observational And Interventional Data (2021)0.00