Meta-learning Bandit Policies By Gradient Ascent

Abstract

Most bandit policies are designed to either minimize regret in any problem instance, making very few assumptions about the underlying environment, or in a Bayesian sense, assuming a prior distribution over environment parameters. The former are often too conservative in practical settings, while the latter require assumptions that are hard to verify in practice. We study bandit problems that fall between these two extremes, where the learning agent has access to sampled bandit instances from an unknown prior distribution \(\mathcal\{P\}\) and aims to achieve high reward on average over the bandit instances drawn from \(\mathcal\{P\}\). This setting is of a particular importance because it lays foundations for meta-learning of bandit policies and reflects more realistic assumptions in many practical domains. We propose the use of parameterized bandit policies that are differentiable and can be optimized using policy gradients. This provides a broadly applicable framework that is easy to

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykveton2020meta

Related papers

• Beyond Variance Reduction: Understanding The True Impact Of Baselines On Policy Optimization (2020)0.00
• Convergence Of A L2 Regularized Policy Gradient Algorithm For The Multi Armed Bandit (2024)0.00
• Policy Gradient Optimization Of Thompson Sampling Policies (2020)0.00
• Off-policy Evaluation And Learning From Logged Bandit Feedback: Error Reduction Via Surrogate Policy (2018)0.00
• Concave Statistical Utility Maximization Bandits Via Influence-function Gradients (2026)0.00
• Learning Optimal Deterministic Policies With Stochastic Policy Gradients (2024)0.00
• Policy Gradient In Partially Observable Environments: Approximation And Convergence (2018)0.00
• Stochastic Gradient Succeeds For Bandits (2024)0.00