Bob And Alice Go To A Bar: Reasoning About Future With Probabilistic Programs

Abstract

It is well known that reinforcement learning can be cast as inference in an appropriate probabilistic model. However, this commonly involves introducing a distribution over agent trajectories with probabilities proportional to exponentiated rewards. In this work, we formulate reinforcement learning as Bayesian inference without resorting to rewards, and show that rewards are derived from agent's preferences, rather than the other way around. We argue that agent preferences should be specified stochastically rather than deterministically. Reinforcement learning via inference with stochastic preferences naturally describes agent behaviors, does not require introducing rewards and exponential weighing of trajectories, and allows to reason about agents using the solid foundation of Bayesian statistics. Stochastic conditioning, a probabilistic programming paradigm for conditioning models on distributions rather than values, is the formalism behind agents with probabilistic preferences. We d

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keytolpin2021bob

Related papers

• Reinforcement Learning And Control As Probabilistic Inference: Tutorial And Review (2018)0.00
• Combining Bayesian Inference And Reinforcement Learning For Agent Decision Making: A Review (2025)0.00
• Inferential Induction: A Novel Framework For Bayesian Reinforcement Learning (2020)0.00
• Control As Probabilistic Inference As An Emergent Communication Mechanism In Multi-agent Reinforcement Learning (2023)0.00
• A Theoretical Connection Between Statistical Physics And Reinforcement Learning (2019)0.00
• Stochastic Reinforcement Learning (2019)5.24
• Probabilistic Recursive Reasoning For Multi-agent Reinforcement Learning (2019)0.00
• Reinforcement Learning With Algorithms From Probabilistic Structure Estimation (2021)0.00