A New Bandit Setting Balancing Information From State Evolution And Corrupted Context

Abstract

We propose a new sequential decision-making setting, combining key aspects of two established online learning problems with bandit feedback. The optimal action to play at any given moment is contingent on an underlying changing state which is not directly observable by the agent. Each state is associated with a context distribution, possibly corrupted, allowing the agent to identify the state. Furthermore, states evolve in a Markovian fashion, providing useful information to estimate the current state via state history. In the proposed problem setting, we tackle the challenge of deciding on which of the two sources of information the agent should base its arm selection. We present an algorithm that uses a referee to dynamically combine the policies of a contextual bandit and a multi-armed bandit. We capture the time-correlation of states through iteratively learning the action-reward transition model, allowing for efficient exploration of actions. Our setting is motivated by adaptive m

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keygalozy2020a

Related papers

• Online Learning With Costly Features In Non-stationary Environments (2023)0.00
• Decision Market Based Learning For Multi-agent Contextual Bandit Problems (2022)0.00
• Multi-action Restless Bandits With Weakly Coupled Constraints: Simultaneous Learning And Control (2024)0.00
• Near-optimal Collaborative Learning In Bandits (2022)0.00
• Bandit Social Learning: Exploration Under Myopic Behavior (2023)0.00
• Provably Efficient Reinforcement Learning For Adversarial Restless Multi-armed Bandits With Unknown Transitions And Bandit Feedback (2024)0.00
• Anytime-valid Off-policy Inference For Contextual Bandits (2022)2.26
• Unified Models Of Human Behavioral Agents In Bandits, Contextual Bandits And RL (2020)8.35