Dynamic Sparsity: Challenging Common Sparsity Assumptions For Learning World Models In Robotic Reinforcement Learning Benchmarks
2025 · Muthukumar Pandaram, Jakob Hollenstein, David Drexel, et al.
Abstract
The use of learned dynamics models, also known as world models, can improve the sample efficiency of reinforcement learning. Recent work suggests that the underlying causal graphs of such dynamics models are sparsely connected, with each of the future state variables depending only on a small subset of the current state variables, and that learning may therefore benefit from sparsity priors. Similarly, temporal sparsity, i.e. sparsely and abruptly changing local dynamics, has also been proposed as a useful inductive bias. In this work, we critically examine these assumptions by analyzing ground-truth dynamics from a set of robotic reinforcement learning environments in the MuJoCo Playground benchmark suite, aiming to determine whether the proposed notions of state and temporal sparsity actually tend to hold in typical reinforcement learning tasks. We study (i) whether the causal graphs of environment dynamics are sparse, (ii) whether such sparsity is state-dependent, and (iii) whet
Authors
(none)
Tags
Stats
Related papers
- Multi-objective Model-based Policy Search For Data-efficient Learning With Sparse Rewards (2018)0.00
- Model-invariant State Abstractions For Model-based Reinforcement Learning (2021)0.00
- Prototypical Context-aware Dynamics Generalization For High-dimensional Model-based Reinforcement Learning (2022)0.00
- Fine-grained Causal Dynamics Learning With Quantization For Improving Robustness In Reinforcement Learning (2024)0.00
- Dynamic Sparse Training For Deep Reinforcement Learning (2021)0.00
- The Utility Of Sparse Representations For Control In Reinforcement Learning (2018)10.48
- Smallworlds: Assessing Dynamics Understanding Of World Models In Isolated Environments (2025)0.00
- Reinforcement Learning Under Latent Dynamics: Toward Statistical And Algorithmic Modularity (2024)0.00