Seeing Is Not Believing: Robust Reinforcement Learning Against Spurious Correlation

Abstract

Robustness has been extensively studied in reinforcement learning (RL) to handle various forms of uncertainty such as random perturbations, rare events, and malicious attacks. In this work, we consider one critical type of robustness against spurious correlation, where different portions of the state do not have correlations induced by unobserved confounders. These spurious correlations are ubiquitous in real-world tasks, for instance, a self-driving car usually observes heavy traffic in the daytime and light traffic at night due to unobservable human activity. A model that learns such useless or even harmful correlation could catastrophically fail when the confounder in the test case deviates from the training one. Although motivated, enabling robustness against spurious correlation poses significant challenges since the uncertainty set, shaped by the unobserved confounder and causal structure, is difficult to characterize and identify. Existing robust algorithms that assume simple an

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyding2023seeing

Related papers

• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• On The Robustness Of Safe Reinforcement Learning Under Observational Perturbations (2022)0.00
• Reccover: Detecting Causal Confusion For Explainable Reinforcement Learning (2022)0.00
• Safe Reinforcement Learning With Dual Robustness (2023)8.60
• Combining Pessimism With Optimism For Robust And Efficient Model-based Deep Reinforcement Learning (2021)0.00
• Sparse Offline Reinforcement Learning With Corruption Robustness (2025)0.00
• Enhancing Robustness In Deep Reinforcement Learning: A Lyapunov Exponent Approach (2024)0.00
• Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, And Generalizability (2022)0.00