Higher : Improving Instruction Following With Hindsight Generation For Experience Replay

Abstract

Language creates a compact representation of the world and allows the description of unlimited situations and objectives through compositionality. While these characterizations may foster instructing, conditioning or structuring interactive agent behavior, it remains an open-problem to correctly relate language understanding and reinforcement learning in even simple instruction following scenarios. This joint learning problem is alleviated through expert demonstrations, auxiliary losses, or neural inductive biases. In this paper, we propose an orthogonal approach called Hindsight Generation for Experience Replay (HIGhER) that extends the Hindsight Experience Replay (HER) approach to the language-conditioned policy setting. Whenever the agent does not fulfill its instruction, HIGhER learns to output a new directive that matches the agent trajectory, and it relabels the episode with a positive reward. To do so, HIGhER learns to map a state into an instruction by using past successful tra

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keycideron2019higher

Related papers

• Sample-efficient Online Learning In LM Agents Via Hindsight Trajectory Rewriting (2025)1.57
• Agenther: Hindsight Experience Replay For LLM Agent Trajectory Relabeling (2026)0.00
• Bias-reduced Hindsight Experience Replay With Virtual Goal Prioritization (2019)9.41
• Hindsight Policy Gradients (2017)0.00
• Improving Experience Replay With Successor Representation (2021)0.00
• Influencing Reinforcement Learning Through Natural Language Guidance (2021)0.00
• Rewriting History With Inverse RL: Hindsight Inference For Policy Improvement (2020)0.00
• Replay For Safety (2021)0.00