Context Meta-reinforcement Learning Via Neuromodulation

Abstract

Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt quickly to tasks from few samples in dynamic environments. Such a feat is achieved through dynamic representations in an agent's policy network (obtained via reasoning about task context, model parameter updates, or both). However, obtaining rich dynamic representations for fast adaptation beyond simple benchmark problems is challenging due to the burden placed on the policy network to accommodate different policies. This paper addresses the challenge by introducing neuromodulation as a modular component to augment a standard policy network that regulates neuronal activities in order to produce efficient dynamic representations for task adaptation. The proposed extension to the policy network is evaluated across multiple discrete and continuous control environments of increasing complexity. To prove the generality and benefits of the extension in meta-RL, the neuromodulated network was applied to two state-of-the-a

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keybeniwhiwhu2021context

Related papers

• Meta-reinforcement Learning With Self-modifying Networks (2022)4.52
• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Distributionally Adaptive Meta Reinforcement Learning (2022)2.26
• A Tutorial On Meta-reinforcement Learning (2023)10.85
• Efficient Off-policy Meta-reinforcement Learning Via Probabilistic Context Variables (2019)0.00
• Reinforcement Learning With Brain-inspired Modulation Can Improve Adaptation To Environmental Changes (2022)0.00
• Lifelong Reinforcement Learning Via Neuromodulation (2024)0.00
• Guided Meta-policy Search (2019)0.00