Robust Asymmetric Learning In Pomdps

Abstract

Policies for partially observed Markov decision processes can be efficiently learned by imitating policies for the corresponding fully observed Markov decision processes. Unfortunately, existing approaches for this kind of imitation learning have a serious flaw: the expert does not know what the trainee cannot see, and so may encourage actions that are sub-optimal, even unsafe, under partial information. We derive an objective to instead train the expert to maximize the expected reward of the imitating agent policy, and use it to construct an efficient algorithm, adaptive asymmetric DAgger (A2D), that jointly trains the expert and the agent. We show that A2D produces an expert policy that the agent can safely imitate, in turn outperforming policies learned by imitating a fixed expert.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywarrington2020robust

Related papers

• Robust Reinforcement Learning In Pomdps With Incomplete And Noisy Observations (2019)0.00
• Sample-efficient Learning Of Pomdps With Multiple Observations In Hindsight (2023)0.00
• Efficient Learning Of Pomdps With Known Observation Model In Average-reward Setting (2024)0.00
• Policy Learning For Robust Markov Decision Process With A Mismatched Generative Model (2022)0.00
• Scaling Internal-state Policy-gradient Methods For Pomdps (2025)0.00
• Efficient Policy Learning For Non-stationary Mdps Under Adversarial Manipulation (2019)0.00
• Finite-state Controllers For (hidden-model) Pomdps Using Deep Reinforcement Learning (2026)0.00
• Agent-state Based Policies In Pomdps: Beyond Belief-state Mdps (2024)0.00