Continuous-time Value Function Approximation In Reproducing Kernel Hilbert Spaces

Abstract

Motivated by the success of reinforcement learning (RL) for discrete-time tasks such as AlphaGo and Atari games, there has been a recent surge of interest in using RL for continuous-time control of physical systems (cf. many challenging tasks in OpenAI Gym and DeepMind Control Suite). Since discretization of time is susceptible to error, it is methodologically more desirable to handle the system dynamics directly in continuous time. However, very few techniques exist for continuous-time RL and they lack flexibility in value function approximation. In this paper, we propose a novel framework for model-based continuous-time value function approximation in reproducing kernel Hilbert spaces. The resulting framework is so flexible that it can accommodate any kind of kernel-based approach, such as Gaussian processes and kernel adaptive filters, and it allows us to handle uncertainties and nonstationarity without prior knowledge about the environment or what basis functions to employ. We demo

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyohnishi2018continuous

Related papers

• Value Function Approximations Via Kernel Embeddings For No-regret Reinforcement Learning (2020)0.00
• Continuous-time Reinforcement Learning: Ellipticity Enables Model-free Value Function Approximation (2026)0.00
• Continuous-time Value Iteration For Multi-agent Reinforcement Learning (2025)0.00
• Deep Radial-basis Value Functions For Continuous Control (2020)0.00
• Offline Reinforcement Learning: Fundamental Barriers For Value Function Approximation (2021)0.00
• Managing Temporal Resolution In Continuous Value Estimation: A Fundamental Trade-off (2022)0.00
• On The Continuity And Smoothness Of The Value Function In Reinforcement Learning And Optimal Control (2024)0.00
• Blending MPC & Value Function Approximation For Efficient Reinforcement Learning (2020)0.00