Mocoda: Model-based Counterfactual Data Augmentation

Abstract

The number of states in a dynamic process is exponential in the number of objects, making reinforcement learning (RL) difficult in complex, multi-object domains. For agents to scale to the real world, they will need to react to and reason about unseen combinations of objects. We argue that the ability to recognize and use local factorization in transition dynamics is a key element in unlocking the power of multi-object reasoning. To this end, we show that (1) known local structure in the environment transitions is sufficient for an exponential reduction in the sample complexity of training a dynamics model, and (2) a locally factored dynamics model provably generalizes out-of-distribution to unseen states and actions. Knowing the local structure also allows us to predict which unseen states and actions this dynamics model will generalize to. We propose to leverage these observations in a novel Model-based Counterfactual Data Augmentation (MoCoDA) framework. MoCoDA applies a learned loc

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keypitis2022mocoda

Related papers

• Sample-efficient Reinforcement Learning Via Counterfactual-based Data Augmentation (2020)0.00
• Understanding When Dynamics-invariant Data Augmentations Benefit Model-free Reinforcement Learning Updates (2023)0.00
• Counterfactual Experience Augmented Off-policy Reinforcement Learning (2025)0.00
• Prototypical Context-aware Dynamics Generalization For High-dimensional Model-based Reinforcement Learning (2022)0.00
• Model-based Offline Reinforcement Learning With Adversarial Data Augmentation (2025)0.00
• Provable RL With Exogenous Distractors Via Multistep Inverse Dynamics (2021)0.00
• Off-policy Reinforcement Learning With Model-based Exploration Augmentation (2025)0.00
• Distributionally Robust Model-based Reinforcement Learning With Large State Spaces (2023)0.00