An MRP Formulation For Supervised Learning: Generalized Temporal Difference Learning Models

Abstract

In traditional statistical learning, data points are usually assumed to be independently and identically distributed (i.i.d.) following an unknown probability distribution. This paper presents a contrasting viewpoint, perceiving data points as interconnected and employing a Markov reward process (MRP) for data modeling. We reformulate the typical supervised learning as an on-policy policy evaluation problem within reinforcement learning (RL), introducing a generalized temporal difference (TD) learning algorithm as a resolution. Theoretically, our analysis establishes connections between the solutions of linear TD learning and ordinary least squares (OLS). Under specific conditions -- particularly when the noise is correlated -- the TD solution serves as a more effective estimator than OLS. Furthermore, we show that when our algorithm is applied with many commonly used loss functions -- such as those found in generalized linear models -- it corresponds to the application of a novel and

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keypan2024an

Related papers

• Discerning Temporal Difference Learning (2023)0.00
• Control Theoretic Analysis Of Temporal Difference Learning (2021)0.00
• Differential Temporal Difference Learning (2018)5.24
• Adaptive Temporal Difference Learning With Linear Function Approximation (2020)0.00
• Loss Dynamics Of Temporal Difference Reinforcement Learning (2023)0.00
• Approximate Temporal Difference Learning Is A Gradient Descent For Reversible Policies (2018)0.00
• Temporal-difference Learning With Nonlinear Function Approximation: Lazy Training And Mean Field Regimes (2019)0.00
• Preferential Temporal Difference Learning (2021)0.00