Meta Reinforcement Learning With Autonomous Inference Of Subtask Dependencies

Abstract

We propose and address a novel few-shot RL problem, where a task is characterized by a subtask graph which describes a set of subtasks and their dependencies that are unknown to the agent. The agent needs to quickly adapt to the task over few episodes during adaptation phase to maximize the return in the test phase. Instead of directly learning a meta-policy, we develop a Meta-learner with Subtask Graph Inference(MSGI), which infers the latent parameter of the task by interacting with the environment and maximizes the return given the latent parameter. To facilitate learning, we adopt an intrinsic reward inspired by upper confidence bound (UCB) that encourages efficient exploration. Our experiment results on two grid-world domains and StarCraft II environments show that the proposed method is able to accurately infer the latent task parameter, and to adapt more efficiently than existing meta RL and hierarchical RL methods.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysohn2020meta

Related papers

• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Boosting Hierarchical Reinforcement Learning With Meta-learning For Complex Task Adaptation (2024)0.00
• Model-based Adversarial Meta-reinforcement Learning (2020)0.00
• Meta-reinforcement Learning Based On Self-supervised Task Representation Learning (2023)7.81
• Hierarchical Meta-reinforcement Learning Via Automated Macro-action Discovery (2024)0.00
• Meta-reinforcement Learning With Universal Policy Adaptation: Provable Near-optimality Under All-task Optimum Comparator (2024)0.00
• Efficient Off-policy Meta-reinforcement Learning Via Probabilistic Context Variables (2019)0.00
• Context Meta-reinforcement Learning Via Neuromodulation (2021)6.34