Guaranteeing Out-of-distribution Detection In Deep RL Via Transition Estimation
2025 · Mohit Prashant, Arvind Easwaran, Suman Das, et al.
Abstract
An issue concerning the use of deep reinforcement learning (RL) agents is whether they can be trusted to perform reliably when deployed, as training environments may not reflect real-life environments. Anticipating instances outside their training scope, learning-enabled systems are often equipped with out-of-distribution (OOD) detectors that alert when a trained system encounters a state it does not recognize or in which it exhibits uncertainty. There exists limited work conducted on the problem of OOD detection within RL, with prior studies being unable to achieve a consensus on the definition of OOD execution within the context of RL. By framing our problem using a Markov Decision Process, we assume there is a transition distribution mapping each state-action pair to another state with some probability. Based on this, we consider the following definition of OOD execution within RL: A transition is OOD if its probability during real-life deployment differs from the transition distrib
Authors
(none)
Tags
Stats
Related papers
- Rethinking Out-of-distribution Detection For Reinforcement Learning: Advancing Methods For Evaluation And Detection (2024)2.26
- Galilai: Out-of-task Distribution Detection Using Causal Active Experimentation For Safe Transfer RL (2021)0.00
- Uncertainty-based Out-of-distribution Detection In Deep Reinforcement Learning (2019)7.50
- Out-of-distribution Dynamics Detection: Rl-relevant Benchmarks And Results (2021)0.00
- Alberdice: Addressing Out-of-distribution Joint Actions In Offline Multi-agent RL Via Alternating Stationary Distribution Correction Estimation (2023)0.00
- Statistical Guarantees For Offline Domain Randomization (2025)0.00
- Sero: Self-supervised Reinforcement Learning For Recovery From Out-of-distribution Situations (2023)3.58
- A Conservative Approach For Few-shot Transfer In Off-dynamics Reinforcement Learning (2023)0.00