Nonparametric Bellman Mappings For Reinforcement Learning: Application To Robust Adaptive Filtering

Abstract

This paper designs novel nonparametric Bellman mappings in reproducing kernel Hilbert spaces (RKHSs) for reinforcement learning (RL). The proposed mappings benefit from the rich approximating properties of RKHSs, adopt no assumptions on the statistics of the data owing to their nonparametric nature, require no knowledge on transition probabilities of Markov decision processes, and may operate without any training data. Moreover, they allow for sampling on-the-fly via the design of trajectory samples, re-use past test data via experience replay, effect dimensionality reduction by random Fourier features, and enable computationally lightweight operations to fit into efficient online or time-adaptive learning. The paper offers also a variational framework to design the free parameters of the proposed Bellman mappings, and shows that appropriate choices of those parameters yield several popular Bellman-mapping designs. As an application, the proposed mappings are employed to offer a novel

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keyakiyama2024nonparametric

Related papers

• Proximal Bellman Mappings For Reinforcement Learning And Their Application To Robust Adaptive Filtering (2023)2.26
• Nonparametric Bellman Mappings For Value Iteration In Distributed Reinforcement Learning (2025)0.00
• Online And Lightweight Kernel-based Approximated Policy Iteration For Dynamic P-norm Linear Adaptive Filtering (2022)0.00
• Continuous-time Value Function Approximation In Reproducing Kernel Hilbert Spaces (2018)0.00
• Computationally Efficient RL Under Linear Bellman Completeness For Deterministic Dynamics (2024)0.00
• Spectral Bellman Method: Unifying Representation And Exploration In RL (2025)0.00
• On Reward-free RL With Kernel And Neural Function Approximations: Single-agent MDP And Markov Game (2021)0.00
• Representation Of Reinforcement Learning Policies In Reproducing Kernel Hilbert Spaces (2020)0.00