Learning Gflownets From Partial Episodes For Improved Convergence And Stability

Abstract

Generative flow networks (GFlowNets) are a family of algorithms for training a sequential sampler of discrete objects under an unnormalized target density and have been successfully used for various probabilistic modeling tasks. Existing training objectives for GFlowNets are either local to states or transitions, or propagate a reward signal over an entire sampling trajectory. We argue that these alternatives represent opposite ends of a gradient bias-variance tradeoff and propose a way to exploit this tradeoff to mitigate its harmful effects. Inspired by the TD(\(\lambda\)) algorithm in reinforcement learning, we introduce subtrajectory balance or SubTB(\(\lambda\)), a GFlowNet training objective that can learn from partial action subsequences of varying lengths. We show that SubTB(\(\lambda\)) accelerates sampler convergence in previously studied and new environments and enables training GFlowNets in environments with longer action sequences and sparser reward landscapes than what wa

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymadan2022learning

Related papers

• Thompson Sampling For Improved Exploration In Gflownets (2023)0.00
• Optimizing Backward Policies In Gflownets Via Trajectory Likelihood Maximization (2024)0.00
• Metagfn: Exploring Distant Modes With Adapted Metadynamics For Continuous Gflownets (2024)0.00
• Temporal Difference Flows (2025)0.00
• Meta Generative Flow Networks With Personalization For Task-specific Adaptation (2023)0.00
• Improved Exploration In Gflownets Via Enhanced Epistemic Neural Networks (2025)0.00
• Guided Flow Policy: Learning From High-value Actions In Offline Reinforcement Learning (2025)0.00
• An Empirical Study Of The Effectiveness Of Using A Replay Buffer On Mode Discovery In Gflownets (2023)0.00