Modeling The Effects Of Environmental And Perceptual Uncertainty Using Deterministic Reinforcement Learning Dynamics With Partial Observability

Abstract

Assessing the systemic effects of uncertainty that arises from agents' partial observation of the true states of the world is critical for understanding a wide range of scenarios. Yet, previous modeling work on agent learning and decision-making either lacks a systematic way to describe this source of uncertainty or puts the focus on obtaining optimal policies using complex models of the world that would impose an unrealistically high cognitive demand on real agents. In this work we aim to efficiently describe the emergent behavior of biologically plausible and parsimonious learning agents faced with partially observable worlds. Therefore we derive and present deterministic reinforcement learning dynamics where the agents observe the true state of the environment only partially. We showcase the broad applicability of our dynamics across different classes of partially observable agent-environment systems. We find that partial observability creates unintuitive benefits in a number of spe

Stats

• citations18
• S2 citations—
• github stars0
• HF likes0
• heat score9.59
• arxiv keybarfuss2021modeling

Related papers

• Uncertainty Maximization In Partially Observable Domains: A Cognitive Perspective (2021)0.00
• Reinforcement Learning Under Partial Observability Guided By Learned Environment Models (2022)6.34
• Optimal Decision-making In Mixed-agent Partially Observable Stochastic Environments Via Reinforcement Learning (2019)0.00
• Probabilistic Inverse Optimal Control For Non-linear Partially Observable Systems Disentangles Perceptual Uncertainty And Behavioral Costs (2023)0.00
• Provably Efficient Reinforcement Learning In Partially Observable Dynamical Systems (2022)0.00
• Understanding Individual Decision-making In Multi-agent Reinforcement Learning: A Dynamical Systems Approach (2025)0.00
• Belief States For Cooperative Multi-agent Reinforcement Learning Under Partial Observability (2025)0.00
• Learning To Forecast Aleatoric And Epistemic Uncertainties Over Long Horizon Trajectories (2023)5.84