Sero: Self-supervised Reinforcement Learning For Recovery From Out-of-distribution Situations

Abstract

Robotic agents trained using reinforcement learning have the problem of taking unreliable actions in an out-of-distribution (OOD) state. Agents can easily become OOD in real-world environments because it is almost impossible for them to visit and learn the entire state space during training. Unfortunately, unreliable actions do not ensure that agents perform their original tasks successfully. Therefore, agents should be able to recognize whether they are in OOD states and learn how to return to the learned state distribution rather than continue to take unreliable actions. In this study, we propose a novel method for retraining agents to recover from OOD situations in a self-supervised manner when they fall into OOD states. Our in-depth experimental results demonstrate that our method substantially improves the agent's ability to recover from OOD situations in terms of sample efficiency and restoration of the performance for the original tasks. Moreover, we show that our method can ret

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keykim2023sero

Related papers

• Rethinking Out-of-distribution Detection For Reinforcement Learning: Advancing Methods For Evaluation And Detection (2024)2.26
• Galilai: Out-of-task Distribution Detection Using Causal Active Experimentation For Safe Transfer RL (2021)0.00
• Uncertainty-based Out-of-distribution Detection In Deep Reinforcement Learning (2019)7.50
• Offline Reinforcement Learning With OOD State Correction And OOD Action Suppression (2024)0.00
• Beyond OOD State Actions: Supported Cross-domain Offline Reinforcement Learning (2023)0.00
• Guaranteeing Out-of-distribution Detection In Deep RL Via Transition Estimation (2025)0.00
• Out-of-distribution Dynamics Detection: Rl-relevant Benchmarks And Results (2021)0.00
• Distributionally Robust Self Paced Curriculum Reinforcement Learning (2025)0.00