Proximal Bellman Mappings For Reinforcement Learning And Their Application To Robust Adaptive Filtering

Abstract

This paper aims at the algorithmic/theoretical core of reinforcement learning (RL) by introducing the novel class of proximal Bellman mappings. These mappings are defined in reproducing kernel Hilbert spaces (RKHSs), to benefit from the rich approximation properties and inner product of RKHSs, they are shown to belong to the powerful Hilbertian family of (firmly) nonexpansive mappings, regardless of the values of their discount factors, and possess ample degrees of design freedom to even reproduce attributes of the classical Bellman mappings and to pave the way for novel RL designs. An approximate policy-iteration scheme is built on the proposed class of mappings to solve the problem of selecting online, at every time instance, the "optimal" exponent \(p\) in a \(p\)-norm loss to combat outliers in linear adaptive filtering, without training data and any knowledge on the statistical properties of the outliers. Numerical tests on synthetic data showcase the superior performance of the p

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyakiyama2023proximal

Related papers

• Nonparametric Bellman Mappings For Reinforcement Learning: Application To Robust Adaptive Filtering (2024)6.34
• Online And Lightweight Kernel-based Approximated Policy Iteration For Dynamic P-norm Linear Adaptive Filtering (2022)0.00
• Distributionally Robust Offline Reinforcement Learning With Linear Function Approximation (2022)0.00
• Nonparametric Bellman Mappings For Value Iteration In Distributed Reinforcement Learning (2025)0.00
• Continuous-time Reinforcement Learning: Ellipticity Enables Model-free Value Function Approximation (2026)0.00
• Minimax Optimal And Computationally Efficient Algorithms For Distributionally Robust Offline Reinforcement Learning (2024)0.00
• Distributionally Robust Off-dynamics Reinforcement Learning: Provable Efficiency With Linear Function Approximation (2024)0.00
• Computationally Efficient RL Under Linear Bellman Completeness For Deterministic Dynamics (2024)0.00