On Assessing The Safety Of Reinforcement Learning Algorithms Using Formal Methods

Abstract

The increasing adoption of Reinforcement Learning in safety-critical systems domains such as autonomous vehicles, health, and aviation raises the need for ensuring their safety. Existing safety mechanisms such as adversarial training, adversarial detection, and robust learning are not always adapted to all disturbances in which the agent is deployed. Those disturbances include moving adversaries whose behavior can be unpredictable by the agent, and as a matter of fact harmful to its learning. Ensuring the safety of critical systems also requires methods that give formal guarantees on the behaviour of the agent evolving in a perturbed environment. It is therefore necessary to propose new solutions adapted to the learning challenges faced by the agent. In this paper, first we generate adversarial agents that exhibit flaws in the agent's policy by presenting moving adversaries. Secondly, We use reward shaping and a modified Q-learning algorithm as defense mechanisms to improve the agent's

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymindom2021on

Related papers

• Certifying Safety In Reinforcement Learning Under Adversarial Perturbation Attacks (2022)0.00
• Rigorous Agent Evaluation: An Adversarial Approach To Uncover Catastrophic Failures (2018)0.00
• Learning To Cope With Adversarial Attacks (2019)0.00
• Safe Reinforcement Learning With Dual Robustness (2023)8.60
• On The Robustness Of Safe Reinforcement Learning Under Observational Perturbations (2022)0.00
• Constrained Black-box Attacks Against Cooperative Multi-agent Reinforcement Learning (2025)0.00
• Improving Robustness Of Reinforcement Learning For Power System Control With Adversarial Training (2021)0.00
• Assured Learning-enabled Autonomy: A Metacognitive Reinforcement Learning Framework (2021)0.00