Reinforcement Learning And Control As Probabilistic Inference: Tutorial And Review

Abstract

The framework of reinforcement learning or optimal control provides a mathematical formalization of intelligent decision making that is powerful and broadly applicable. While the general form of the reinforcement learning problem enables effective reasoning about uncertainty, the connection between reinforcement learning and inference in probabilistic models is not immediately obvious. However, such a connection has considerable value when it comes to algorithm design: formalizing a problem as probabilistic inference in principle allows us to bring to bear a wide array of approximate inference tools, extend the model in flexible and powerful ways, and reason about compositionality and partial observability. In this article, we will discuss how a generalization of the reinforcement learning or optimal control problem, which is sometimes termed maximum entropy reinforcement learning, is equivalent to exact probabilistic inference in the case of deterministic dynamics, and variational inf

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylevine2018reinforcement

Related papers

• From Reinforcement Learning To Optimal Control: A Unified Framework For Sequential Decisions (2019)0.00
• Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach (2023)0.00
• Variational Inference For Model-free And Model-based Reinforcement Learning (2022)0.00
• VIREL: A Variational Inference Framework For Reinforcement Learning (2018)0.00
• Probabilistic Inverse Optimal Control For Non-linear Partially Observable Systems Disentangles Perceptual Uncertainty And Behavioral Costs (2023)0.00
• A Tour Of Reinforcement Learning: The View From Continuous Control (2018)19.86
• Combining Bayesian Inference And Reinforcement Learning For Agent Decision Making: A Review (2025)0.00
• Bob And Alice Go To A Bar: Reasoning About Future With Probabilistic Programs (2021)0.00