Bayesian Decision Making Around Experts

Abstract

Complex learning agents are increasingly deployed alongside existing experts, such as human operators or previously trained agents. However, it remains unclear how should learners optimally incorporate certain forms of expert data, which may differ in structure from the learner's own action-outcome experiences. We study this problem in the context of Bayesian multi-armed bandits, considering: (i) offline settings, where the learner receives a dataset of outcomes from the expert's optimal policy before interaction, and (ii) simultaneous settings, where the learner must choose at each step whether to update its beliefs based on its own experience, or based on the outcome simultaneously achieved by an expert. We formalize how expert data influences the learner's posterior, and prove that pretraining on expert outcomes tightens information-theoretic regret bounds by the mutual information between the expert data and the optimal action. For the simultaneous setting, we propose an informatio

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyornia2025bayesian

Related papers

• A New Bandit Setting Balancing Information From State Evolution And Corrupted Context (2020)0.00
• Decision Market Based Learning For Multi-agent Contextual Bandit Problems (2022)0.00
• Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning With Clairvoyant Experts (2020)0.00
• Online Learning With Costly Features In Non-stationary Environments (2023)0.00
• Bayesian Bandits: Balancing The Exploration-exploitation Tradeoff Via Double Sampling (2017)0.00
• Pure Exploration Under Mediators' Feedback (2023)0.00
• A Bayesian Solution To The Imitation Gap (2024)0.00
• Learning Equilibria From Data: Provably Efficient Multi-agent Imitation Learning (2025)0.00