Controllable Flow Matching For Online Reinforcement Learning

Abstract

Model-based reinforcement learning (MBRL) typically relies on modeling environment dynamics for data efficiency. However, due to the accumulation of model errors over long-horizon rollouts, such methods often face challenges in maintaining modeling stability. To address this, we propose CtrlFlow, a trajectory-level synthetic method using conditional flow matching (CFM), which directly modeling the distribution of trajectories from initial states to high-return terminal states without explicitly modeling the environment transition function. Our method ensures optimal trajectory sampling by minimizing the control energy governed by the non-linear Controllability Gramian Matrix, while the generated diverse trajectory data significantly enhances the robustness and cross-task generalization of policy learning. In online settings, CtrlFlow demonstrates the better performance on common MuJoCo benchmark tasks than dynamics models and achieves superior sample efficiency compared to standard MBR

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2025controllable

Related papers

• Composite Flow Matching For Reinforcement Learning With Shifted-dynamics Data (2025)0.00
• FM-IRL: Flow-matching For Reward Modeling And Policy Regularization In Reinforcement Learning (2025)0.00
• Reverse Flow Matching: A Unified Framework For Online Reinforcement Learning With Diffusion And Flow Policies (2026)0.00
• Evolving Diffusion And Flow Matching Policies For Online Reinforcement Learning (2025)0.00
• Planning With Exploration: Addressing Dynamics Bottleneck In Model-based Reinforcement Learning (2020)0.00
• Flowpg: Action-constrained Policy Gradient With Normalizing Flows (2024)0.00
• Guided Flow Policy: Learning From High-value Actions In Offline Reinforcement Learning (2025)0.00
• Max-entropy Reinforcement Learning With Flow Matching And A Case Study On LQR (2025)0.00