Galilai: Out-of-task Distribution Detection Using Causal Active Experimentation For Safe Transfer RL

Abstract

Out-of-distribution (OOD) detection is a well-studied topic in supervised learning. Extending the successes in supervised learning methods to the reinforcement learning (RL) setting, however, is difficult due to the data generating process - RL agents actively query their environment for data, and the data are a function of the policy followed by the agent. An agent could thus neglect a shift in the environment if its policy did not lead it to explore the aspect of the environment that shifted. Therefore, to achieve safe and robust generalization in RL, there exists an unmet need for OOD detection through active experimentation. Here, we attempt to bridge this lacuna by first defining a causal framework for OOD scenarios or environments encountered by RL agents in the wild. Then, we propose a novel task: that of Out-of-Task Distribution (OOTD) detection. We introduce an RL agent that actively experiments in a test environment and subsequently concludes whether it is OOTD or not. We nam

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysontakke2021galilai

Related papers

• Rethinking Out-of-distribution Detection For Reinforcement Learning: Advancing Methods For Evaluation And Detection (2024)2.26
• Guaranteeing Out-of-distribution Detection In Deep RL Via Transition Estimation (2025)0.00
• Out-of-distribution Dynamics Detection: Rl-relevant Benchmarks And Results (2021)0.00
• Sero: Self-supervised Reinforcement Learning For Recovery From Out-of-distribution Situations (2023)3.58
• Uncertainty-based Out-of-distribution Detection In Deep Reinforcement Learning (2019)7.50
• An Information-theoretic Analysis Of OOD Generalization In Meta-reinforcement Learning (2025)0.00
• Alberdice: Addressing Out-of-distribution Joint Actions In Offline Multi-agent RL Via Alternating Stationary Distribution Correction Estimation (2023)0.00
• Planning To Go Out-of-distribution In Offline-to-online Reinforcement Learning (2023)0.00