Combining Parametric And Nonparametric Models For Off-policy Evaluation

Abstract

We consider a model-based approach to perform batch off-policy evaluation in reinforcement learning. Our method takes a mixture-of-experts approach to combine parametric and non-parametric models of the environment such that the final value estimate has the least expected error. We do so by first estimating the local accuracy of each model and then using a planner to select which model to use at every time step as to minimize the return error estimate along entire trajectories. Across a variety of domains, our mixture-based approach outperforms the individual models alone as well as state-of-the-art importance sampling-based estimators.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygottesman2019combining

Related papers

• Optimal Mixture Weights For Off-policy Evaluation With Multiple Behavior Policies (2020)0.00
• Bootstrapping With Models: Confidence Intervals For Off-policy Evaluation (2016)9.23
• Infinite-horizon Off-policy Policy Evaluation With Multiple Behavior Policies (2019)0.00
• Minimax-optimal Off-policy Evaluation With Linear Function Approximation (2020)0.00
• Low Variance Off-policy Evaluation With State-based Importance Sampling (2022)0.00
• Probabilistic Mixture-of-experts For Efficient Deep Reinforcement Learning (2021)0.00
• Batch Reinforcement Learning With A Nonparametric Off-policy Policy Gradient (2020)0.00
• Towards Optimal Off-policy Evaluation For Reinforcement Learning With Marginalized Importance Sampling (2019)0.00