Logarithmic Smoothing For Adaptive Pac-bayesian Off-policy Learning

Abstract

Off-policy learning serves as the primary framework for learning optimal policies from logged interactions collected under a static behavior policy. In this work, we investigate the more practical and flexible setting of adaptive off-policy learning, where policies are iteratively refined and re-deployed to collect higher-quality data. Building on the success of PAC-Bayesian learning with Logarithmic Smoothing (LS) in static settings, we extend this framework to the adaptive scenario using tools from online PAC-Bayesian theory. Furthermore, we demonstrate that a principled adjustment to the LS estimator naturally accommodates multiple rounds of deployment and yields faster convergence rates under mild conditions. Our method matches the performance of leading offline approaches in static settings, and significantly outperforms them when intermediate policy deployments are allowed. Empirical evaluations across diverse scenarios highlight both the advantages of adaptive data collection an

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhaddouche2025logarithmic

Related papers

• Logarithmic Smoothing For Pessimistic Off-policy Evaluation, Selection And Learning (2024)0.00
• Policy Learning With Adaptively Collected Data (2021)0.00
• Off-policy Evaluation And Learning From Logged Bandit Feedback: Error Reduction Via Surrogate Policy (2018)0.00
• Policy Learning "without" Overlap: Pessimism And Generalized Empirical Bernstein's Inequality (2022)0.00
• Log-sum-exponential Estimator For Off-policy Evaluation And Learning (2025)0.00
• Offline-boosted Actor-critic: Adaptively Blending Optimal Historical Behaviors In Deep Off-policy RL (2024)0.00
• Adaptive Layerwise Perturbation: Unifying Off-policy Corrections For LLM RL (2026)0.00
• Adaptive Doubly Robust Estimator From Non-stationary Logging Policy Under A Convergence Of Average Probability (2021)0.00