An Efficient, Expressive And Local Minima-free Method For Learning Controlled Dynamical Systems

Abstract

We propose a framework for modeling and estimating the state of controlled dynamical systems, where an agent can affect the system through actions and receives partial observations. Based on this framework, we propose the Predictive State Representation with Random Fourier Features (RFFPSR). A key property in RFF-PSRs is that the state estimate is represented by a conditional distribution of future observations given future actions. RFF-PSRs combine this representation with moment-matching, kernel embedding and local optimization to achieve a method that enjoys several favorable qualities: It can represent controlled environments which can be affected by actions; it has an efficient and theoretically justified learning algorithm; it uses a non-parametric representation that has expressive power to represent continuous non-linear dynamics. We provide a detailed formulation, a theoretical analysis and an experimental evaluation that demonstrates the effectiveness of our method.

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyhefny2017an

Related papers

• Adaptive Input Estimation In Linear Dynamical Systems With Applications To Learning-from-observations (2018)0.00
• Sample Complexity Of Estimating The Policy Gradient For Nearly Deterministic Dynamical Systems (2019)0.00
• Optimal Control Of Probabilistic Dynamics Models Via Mean Hamiltonian Minimization (2025)0.00
• Optimal Exploration For Model-based RL In Nonlinear Systems (2023)0.00
• Model-based Reinforcement Learning For Control Under Time-varying Dynamics (2026)0.00
• Prediction And Control With Temporal Segment Models (2017)0.00
• Multi-fidelity Reinforcement Learning Control For Complex Dynamical Systems (2025)0.00
• Provably Efficient Reinforcement Learning In Partially Observable Dynamical Systems (2022)0.00