Flow To Control: Offline Reinforcement Learning With Lossless Primitive Discovery

Abstract

Offline reinforcement learning (RL) enables the agent to effectively learn from logged data, which significantly extends the applicability of RL algorithms in real-world scenarios where exploration can be expensive or unsafe. Previous works have shown that extracting primitive skills from the recurring and temporally extended structures in the logged data yields better learning. However, these methods suffer greatly when the primitives have limited representation ability to recover the original policy space, especially in offline settings. In this paper, we give a quantitative characterization of the performance of offline hierarchical learning and highlight the importance of learning lossless primitives. To this end, we propose to use a *flow*-based structure as the representation for low-level policies. This allows us to represent the behaviors in the dataset faithfully while keeping the expression ability to recover the whole policy space. We show that such lossless primitives can d

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyyang2022flow

Related papers

• OPAL: Offline Primitive Discovery For Accelerating Offline Reinforcement Learning (2020)0.00
• Expressive Value Learning For Scalable Offline Reinforcement Learning (2025)0.00
• A Workflow For Offline Model-free Robotic Reinforcement Learning (2021)0.00
• Guided Flow Policy: Learning From High-value Actions In Offline Reinforcement Learning (2025)0.00
• Composite Flow Matching For Reinforcement Learning With Shifted-dynamics Data (2025)0.00
• Reverse Flow Matching: A Unified Framework For Online Reinforcement Learning With Diffusion And Flow Policies (2026)0.00
• FM-IRL: Flow-matching For Reward Modeling And Policy Regularization In Reinforcement Learning (2025)0.00
• Evolving Diffusion And Flow Matching Policies For Online Reinforcement Learning (2025)0.00