Logarithmic Smoothing For Pessimistic Off-policy Evaluation, Selection And Learning

Abstract

This work investigates the offline formulation of the contextual bandit problem, where the goal is to leverage past interactions collected under a behavior policy to evaluate, select, and learn new, potentially better-performing, policies. Motivated by critical applications, we move beyond point estimators. Instead, we adopt the principle of pessimism where we construct upper bounds that assess a policy's worst-case performance, enabling us to confidently select and learn improved policies. Precisely, we introduce novel, fully empirical concentration bounds for a broad class of importance weighting risk estimators. These bounds are general enough to cover most existing estimators and pave the way for the development of new ones. In particular, our pursuit of the tightest bound within this class motivates a novel estimator (LS), that logarithmically smooths large importance weights. The bound for LS is provably tighter than its competitors, and naturally results in improved policy selec

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysakhi2024logarithmic

Related papers

• Logarithmic Smoothing For Adaptive Pac-bayesian Off-policy Learning (2025)0.00
• Off-policy Evaluation And Learning From Logged Bandit Feedback: Error Reduction Via Surrogate Policy (2018)0.00
• Log-sum-exponential Estimator For Off-policy Evaluation And Learning (2025)0.00
• Policy Learning "without" Overlap: Pessimism And Generalized Empirical Bernstein's Inequality (2022)0.00
• Is Pessimism Provably Efficient For Offline RL? (2020)0.00
• Doubly Robust Interval Estimation For Optimal Policy Evaluation In Online Learning (2021)0.00
• Pessimistic Q-learning For Offline Reinforcement Learning: Towards Optimal Sample Complexity (2022)0.00
• Bellman-consistent Pessimism For Offline Reinforcement Learning (2021)0.00