Out-of-distribution Dynamics Detection: Rl-relevant Benchmarks And Results

Abstract

We study the problem of out-of-distribution dynamics (OODD) detection, which involves detecting when the dynamics of a temporal process change compared to the training-distribution dynamics. This is relevant to applications in control, reinforcement learning (RL), and multi-variate time-series, where changes to test time dynamics can impact the performance of learning controllers/predictors in unknown ways. This problem is particularly important in the context of deep RL, where learned controllers often overfit to the training environment. Currently, however, there is a lack of established OODD benchmarks for the types of environments commonly used in RL research. Our first contribution is to design a set of OODD benchmarks derived from common RL environments with varying types and intensities of OODD. Our second contribution is to design a strong OODD baseline approach based on recurrent implicit quantile network (RIQN), which monitors autoregressive prediction errors for OODD detecti

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydanesh2021out

Related papers

• Rethinking Out-of-distribution Detection For Reinforcement Learning: Advancing Methods For Evaluation And Detection (2024)2.26
• Galilai: Out-of-task Distribution Detection Using Causal Active Experimentation For Safe Transfer RL (2021)0.00
• Guaranteeing Out-of-distribution Detection In Deep RL Via Transition Estimation (2025)0.00
• Uncertainty-based Out-of-distribution Detection In Deep Reinforcement Learning (2019)7.50
• Assessing The Impact Of Distribution Shift On Reinforcement Learning Performance (2024)0.00
• A Conservative Approach For Few-shot Transfer In Off-dynamics Reinforcement Learning (2023)0.00
• Statistical Guarantees For Offline Domain Randomization (2025)0.00
• Sero: Self-supervised Reinforcement Learning For Recovery From Out-of-distribution Situations (2023)3.58