Memory Sequence Length Of Data Sampling Impacts The Adaptation Of Meta-reinforcement Learning Agents

Abstract

Fast adaptation to new tasks is extremely important for embodied agents in the real world. Meta-reinforcement learning (meta-RL) has emerged as an effective method to enable fast adaptation in unknown environments. Compared to on-policy meta-RL algorithms, off-policy algorithms rely heavily on efficient data sampling strategies to extract and represent the historical trajectories. However, little is known about how different data sampling methods impact the ability of meta-RL agents to represent unknown environments. Here, we investigate the impact of data sampling strategies on the exploration and adaptability of meta-RL agents. Specifically, we conducted experiments with two types of off-policy meta-RL algorithms based on Thompson sampling and Bayes-optimality theories in continuous control tasks within the MuJoCo environment and sparse reward navigation tasks. Our analysis revealed the long-memory and short-memory sequence sampling strategies affect the representation and adaptive c

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyzhang2024memory

Related papers

• Efficient Meta Reinforcement Learning For Preference-based Fast Adaptation (2022)0.00
• Sampling Attacks On Meta Reinforcement Learning: A Minimax Formulation And Complexity Analysis (2022)0.00
• Context Meta-reinforcement Learning Via Neuromodulation (2021)6.34
• Distributionally Adaptive Meta Reinforcement Learning (2022)2.26
• Higher Replay Ratio Empowers Sample-efficient Multi-agent Reinforcement Learning (2024)0.00
• On The Effectiveness Of Fine-tuning Versus Meta-reinforcement Learning (2022)0.00
• External Model Motivated Agents: Reinforcement Learning For Enhanced Environment Sampling (2024)0.00
• Harnessing The Power Of Reinforcement Learning For Adaptive MCMC (2025)0.00