Sparse Offline Reinforcement Learning With Corruption Robustness

Abstract

We investigate robustness to strong data corruption in offline sparse reinforcement learning (RL). In our setting, an adversary may arbitrarily perturb a fraction of the collected trajectories from a high-dimensional but sparse Markov decision process, and our goal is to estimate a near optimal policy. The main challenge is that, in the high-dimensional regime where the number of samples \(N\) is smaller than the feature dimension \(d\), exploiting sparsity is essential for obtaining non-vacuous guarantees but has not been systematically studied in offline RL. We analyse the problem under uniform coverage and sparse single-concentrability assumptions. While Least Square Value Iteration (LSVI), a standard approach for robust offline RL, performs well under uniform coverage, we show that integrating sparsity into LSVI is unnatural, and its analysis may break down due to overly pessimistic bonuses. To overcome this, we propose actor-critic methods with sparse robust estimator oracles, whi

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytran2025sparse

Related papers

• Corruption Robust Offline Reinforcement Learning With Human Feedback (2024)0.00
• Towards Robust Offline Reinforcement Learning Under Diverse Data Corruption (2023)0.00
• Corruption-robust Offline Two-player Zero-sum Markov Games (2024)0.00
• Enhancing Robustness Of Offline Reinforcement Learning Under Data Corruption Via Sharpness-aware Minimization (2025)0.00
• Uncertainty-based Offline Variational Bayesian Reinforcement Learning For Robustness Under Diverse Data Corruptions (2024)2.26
• Towards Robust Policy: Enhancing Offline Reinforcement Learning With Adversarial Attacks And Defenses (2024)3.58
• Distributionally Robust Model-based Offline Reinforcement Learning With Near-optimal Sample Complexity (2022)0.00
• Distributionally Robust Offline Reinforcement Learning With Linear Function Approximation (2022)0.00