Reinforcement Learning With Perturbed Rewards

Abstract

Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, a deep reinforcement learning (DRL) algorithm can be manipulated to produce arbitrary errors by receiving corrupted rewards. In this paper, we consider noisy RL problems with perturbed rewards, which can be approximated with a confusion matrix. We develop a robust RL framework that enables agents to learn in noisy environments where only perturbed rewards are observed. Our solution framework builds on existing RL/DRL algorithms and firstly addresses the biased noisy reward setting without any assumptions on the true distribution (e.g., zero-mean Gaussian noise as made in previous works). The core ideas of our solution include estimating a reward confusion matrix and defin

Stats

• citations67
• S2 citations—
• github stars0
• HF likes0
• heat score13.74
• arxiv keywang2018reinforcement

Related papers

• The Distributional Reward Critic Framework For Reinforcement Learning Under Perturbed Rewards (2024)0.00
• Disturbing Reinforcement Learning Agents With Corrupted Rewards (2021)0.00
• Regret-based Defense In Adversarial Reinforcement Learning (2023)0.00
• Robust Reinforcement Learning On State Observations With Learned Optimal Adversary (2021)0.00
• Adaptive Symmetric Reward Noising For Reinforcement Learning (2019)0.00
• Quantifying First-order Markov Violations In Noisy Reinforcement Learning: A Causal Discovery Approach (2025)0.00
• Exploring The Training Robustness Of Distributional Reinforcement Learning Against Noisy State Observations (2021)0.00
• Robust Deep Reinforcement Learning With Adaptive Adversarial Perturbations In Action Space (2024)6.20