Feasible Adversarial Robust Reinforcement Learning For Underspecified Environments

Abstract

Robust reinforcement learning (RL) considers the problem of learning policies that perform well in the worst case among a set of possible environment parameter values. In real-world environments, choosing the set of possible values for robust RL can be a difficult task. When that set is specified too narrowly, the agent will be left vulnerable to reasonable parameter values unaccounted for. When specified too broadly, the agent will be too cautious. In this paper, we propose Feasible Adversarial Robust RL (FARR), a novel problem formulation and objective for automatically determining the set of environment parameter values over which to be robust. FARR implicitly defines the set of feasible parameter values as those on which an agent could achieve a benchmark reward given enough training resources. By formulating this problem as a two-player zero-sum game, optimizing the FARR objective jointly produces an adversarial distribution over parameter values with feasible support and a policy

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylanier2022feasible

Related papers

• Safe Reinforcement Learning With Dual Robustness (2023)8.60
• Sample-efficient Robust Multi-agent Reinforcement Learning In The Face Of Environmental Uncertainty (2024)0.00
• Robust Adversarial Reinforcement Learning Via Bounded Rationality Curricula (2023)0.00
• Efficient Adversarial Training Without Attacking: Worst-case-aware Robust Reinforcement Learning (2022)0.00
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Robust Model-based Reinforcement Learning With An Adversarial Auxiliary Model (2024)0.00
• Robust Cooperative Multi-agent Reinforcement Learning:a Mean-field Type Game Perspective (2024)0.00
• On The Robustness Of Safe Reinforcement Learning Under Observational Perturbations (2022)0.00