Backpropagation Through Time And Space: Learning Numerical Methods With Multi-agent Reinforcement Learning

Abstract

We introduce Backpropagation Through Time and Space (BPTTS), a method for training a recurrent spatio-temporal neural network, that is used in a homogeneous multi-agent reinforcement learning (MARL) setting to learn numerical methods for hyperbolic conservation laws. We treat the numerical schemes underlying partial differential equations (PDEs) as a Partially Observable Markov Game (POMG) in Reinforcement Learning (RL). Similar to numerical solvers, our agent acts at each discrete location of a computational space for efficient and generalizable learning. To learn higher-order spatial methods by acting on local states, the agent must discern how its actions at a given spatiotemporal location affect the future evolution of the state. The manifestation of this non-stationarity is addressed by BPTTS, which allows for the flow of gradients across both space and time. The learned numerical policies are comparable to the SOTA numerics in two settings, the Burgers' Equation and the Euler Equ

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyway2022backpropagation

Related papers

• Non-stationary Policy Learning For Multi-timescale Multi-agent Reinforcement Learning (2023)5.24
• Hierarchical Deep Multiagent Reinforcement Learning With Temporal Abstraction (2018)0.00
• Tesseract: Tensorised Actors For Multi-agent Reinforcement Learning (2021)0.00
• Multi-agent Reinforcement Learning In Stochastic Networked Systems (2020)0.00
• Fast Multi-agent Temporal-difference Learning Via Homotopy Stochastic Primal-dual Optimization (2019)0.00
• Continuous-time Value Iteration For Multi-agent Reinforcement Learning (2025)0.00
• Real-time Recurrent Reinforcement Learning (2023)2.26
• Preference-based Multi-agent Reinforcement Learning: Data Coverage And Algorithmic Techniques (2024)0.00