Deep Radial-basis Value Functions For Continuous Control

Abstract

A core operation in reinforcement learning (RL) is finding an action that is optimal with respect to a learned value function. This operation is often challenging when the learned value function takes continuous actions as input. We introduce deep radial-basis value functions (RBVFs): value functions learned using a deep network with a radial-basis function (RBF) output layer. We show that the maximum action-value with respect to a deep RBVF can be approximated easily and accurately. Moreover, deep RBVFs can represent any true value function owing to their support for universal function approximation. We extend the standard DQN algorithm to continuous control by endowing the agent with a deep RBVF. We show that the resultant agent, called RBF-DQN, significantly outperforms value-function-only baselines, and is competitive with state-of-the-art actor-critic algorithms.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyasadi2020deep

Related papers

• Broad Critic Deep Actor Reinforcement Learning For Continuous Control (2024)0.00
• Parameter-based Value Functions (2020)2.26
• Continuous-time Value Function Approximation In Reproducing Kernel Hilbert Spaces (2018)0.00
• Approximating Two Value Functions Instead Of One: Towards Characterizing A New Family Of Deep Reinforcement Learning Algorithms (2019)0.00
• Deep Exploration With Pac-bayes (2024)0.00
• Convex Programs And Lyapunov Functions For Reinforcement Learning: A Unified Perspective On The Analysis Of Value-based Methods (2022)2.26
• Solving Continuous Control Via Q-learning (2022)0.00
• Mixed Q-functionals: Advancing Value-based Methods In Cooperative MARL With Continuous Action Domains (2024)0.00