AKF-SR: Adaptive Kalman Filtering-based Successor Representation

Abstract

Recent studies in neuroscience suggest that Successor Representation (SR)-based models provide adaptation to changes in the goal locations or reward function faster than model-free algorithms, together with lower computational cost compared to that of model-based algorithms. However, it is not known how such representation might help animals to manage uncertainty in their decision-making. Existing methods for SR learning do not capture uncertainty about the estimated SR. In order to address this issue, the paper presents a Kalman filter-based SR framework, referred to as Adaptive Kalman Filtering-based Successor Representation (AKF-SR). First, Kalman temporal difference approach, which is a combination of the Kalman filter and the temporal difference method, is used within the AKF-SR framework to cast the SR learning procedure into a filtering problem to benefit from the uncertainty estimation of the SR, and also decreases in memory requirement and sensitivity to model's parameters in

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keymalekzadeh2022akf

Related papers

• A Neurally Plausible Model Learns Successor Representations In Partially Observable Environments (2019)0.00
• Successor Feature Representations (2021)0.00
• Multi-agent Reinforcement Learning Via Adaptive Kalman Temporal Difference And Successor Representation (2021)0.00
• A First-occupancy Representation For Reinforcement Learning (2021)0.00
• Successor Features Combine Elements Of Model-free And Model-based Reinforcement Learning (2019)0.00
• Successor Uncertainties: Exploration And Uncertainty In Temporal Difference Learning (2018)0.00
• Uncertainty-aware Transfer Across Tasks Using Hybrid Model-based Successor Feature Reinforcement Learning (2023)5.24
• MM-KTD: Multiple Model Kalman Temporal Differences For Reinforcement Learning (2020)0.00