Sample And Computationally Efficient Continuous-time Reinforcement Learning With General Function Approximation

Abstract

Continuous-time reinforcement learning (CTRL) provides a principled framework for sequential decision-making in environments where interactions evolve continuously over time. Despite its empirical success, the theoretical understanding of CTRL remains limited, especially in settings with general function approximation. In this work, we propose a model-based CTRL algorithm that achieves both sample and computational efficiency. Our approach leverages optimism-based confidence sets to establish the first sample complexity guarantee for CTRL with general function approximation, showing that a near-optimal policy can be learned with a suboptimality gap of \(\tilde\{O\}(\sqrt\{d_\{\mathcal\{R\}\} + d_\{\mathcal\{F\}\}\}N^\{-1/2\})\) using \(N\) measurements, where \(d_\{\mathcal\{R\}\}\) and \(d_\{\mathcal\{F\}\}\) denote the distributional Eluder dimensions of the reward and dynamic functions, respectively, capturing the complexity of general function approximation in reinforcement learnin

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhao2025sample

Related papers

• A Nearly Optimal And Low-switching Algorithm For Reinforcement Learning With General Function Approximation (2023)0.00
• Online Sub-sampling For Reinforcement Learning With General Function Approximation (2021)0.00
• Uniform-pac Bounds For Reinforcement Learning With Linear Function Approximation (2021)0.00
• Posterior Sampling With Delayed Feedback For Reinforcement Learning With Linear Function Approximation (2023)0.00
• Optimal Conservative Offline RL With General Function Approximation Via Augmented Lagrangian (2022)0.00
• RL In Markov Games With Independent Function Approximation: Improved Sample Complexity Bound Under The Local Access Model (2024)0.00
• Continuous-time Value Function Approximation In Reproducing Kernel Hilbert Spaces (2018)0.00
• Provably Efficient Reinforcement Learning With Linear Function Approximation (2019)11.76