Selective Credit Assignment

Abstract

Efficient credit assignment is essential for reinforcement learning algorithms in both prediction and control settings. We describe a unified view on temporal-difference algorithms for selective credit assignment. These selective algorithms apply weightings to quantify the contribution of learning updates. We present insights into applying weightings to value-based learning and planning algorithms, and describe their role in mediating the backward credit distribution in prediction and control. Within this space, we identify some existing online learning algorithms that can assign credit selectively as special cases, as well as add new algorithms that assign credit backward in time counterfactually, allowing credit to be assigned off-trajectory and off-policy.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychelu2022selective

Related papers

• Counterfactual Credit Assignment In Model-free Reinforcement Learning (2020)0.00
• An Information-theoretic Perspective On Credit Assignment In Reinforcement Learning (2021)0.00
• Adaptive Pairwise Weights For Temporal Credit Assignment (2021)0.00
• A Survey Of Temporal Credit Assignment In Deep Reinforcement Learning (2023)0.00
• Modularity In Reinforcement Learning Via Algorithmic Independence In Credit Assignment (2021)0.00
• Sequence Compression Speeds Up Credit Assignment In Reinforcement Learning (2024)0.00
• Asynchronous Credit Assignment For Multi-agent Reinforcement Learning (2024)0.00
• Demystifying The Recency Heuristic In Temporal-difference Learning (2024)0.00