DEIR: Efficient And Robust Exploration Through Discriminative-model-based Episodic Intrinsic Rewards

Abstract

Exploration is a fundamental aspect of reinforcement learning (RL), and its effectiveness is a deciding factor in the performance of RL algorithms, especially when facing sparse extrinsic rewards. Recent studies have shown the effectiveness of encouraging exploration with intrinsic rewards estimated from novelties in observations. However, there is a gap between the novelty of an observation and an exploration, as both the stochasticity in the environment and the agent's behavior may affect the observation. To evaluate exploratory behaviors accurately, we propose DEIR, a novel method in which we theoretically derive an intrinsic reward with a conditional mutual information term that principally scales with the novelty contributed by agent explorations, and then implement the reward with a discriminative forward model. Extensive experiments on both standard and advanced exploration tasks in MiniGrid show that DEIR quickly learns a better policy than the baselines. Our evaluations on Pro

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywan2023deir

Related papers

• Rewarding Episodic Visitation Discrepancy For Exploration In Reinforcement Learning (2022)0.00
• MIR: Efficient Exploration In Episodic Multi-agent Reinforcement Learning Via Mutual Intrinsic Reward (2025)0.00
• Decoupled Reinforcement Learning To Stabilise Intrinsically-motivated Exploration (2021)2.26
• Exploration Via Elliptical Episodic Bonuses (2022)3.58
• The Impact Of Intrinsic Rewards On Exploration In Reinforcement Learning (2025)0.00
• Redeeming Intrinsic Rewards Via Constrained Optimization (2022)0.00
• Information Content Exploration (2023)0.00
• Episodic Reinforcement Learning With Expanded State-reward Space (2024)0.00