Towards Robust Offline Reinforcement Learning Under Diverse Data Corruption

Abstract

Offline reinforcement learning (RL) presents a promising approach for learning reinforced policies from offline datasets without the need for costly or unsafe interactions with the environment. However, datasets collected by humans in real-world environments are often noisy and may even be maliciously corrupted, which can significantly degrade the performance of offline RL. In this work, we first investigate the performance of current offline RL algorithms under comprehensive data corruption, including states, actions, rewards, and dynamics. Our extensive experiments reveal that implicit Q-learning (IQL) demonstrates remarkable resilience to data corruption among various offline RL algorithms. Furthermore, we conduct both empirical and theoretical analyses to understand IQL's robust performance, identifying its supervised policy learning scheme as the key factor. Despite its relative robustness, IQL still suffers from heavy-tail targets of Q functions under dynamics corruption. To tack

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2023towards

Related papers

• Corruption Robust Offline Reinforcement Learning With Human Feedback (2024)0.00
• Sparse Offline Reinforcement Learning With Corruption Robustness (2025)0.00
• Enhancing Robustness Of Offline Reinforcement Learning Under Data Corruption Via Sharpness-aware Minimization (2025)0.00
• Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses (2024)3.58
• Robust Offline Reinforcement Learning With Gradient Penalty And Constraint Relaxation (2022)0.00
• Uncertainty-based Offline Variational Bayesian Reinforcement Learning For Robustness Under Diverse Data Corruptions (2024)2.26
• An Optimistic Perspective On Offline Reinforcement Learning (2019)0.00
• Boosting Offline Reinforcement Learning With Residual Generative Modeling (2021)0.00