Learning To Forecast Aleatoric And Epistemic Uncertainties Over Long Horizon Trajectories

Abstract

Giving autonomous agents the ability to forecast their own outcomes and uncertainty will allow them to communicate their competencies and be used more safely. We accomplish this by using a learned world model of the agent system to forecast full agent trajectories over long time horizons. Real world systems involve significant sources of both aleatoric and epistemic uncertainty that compound and interact over time in the trajectory forecasts. We develop a deep generative world model that quantifies aleatoric uncertainty while incorporating the effects of epistemic uncertainty during the learning process. We show on two reinforcement learning problems that our uncertainty model produces calibrated outcome uncertainty estimates over the full trajectory horizon.

Stats

• citations5
• S2 citations—
• github stars0
• HF likes0
• heat score5.84
• arxiv keyacharya2023learning

Related papers

• Modeling The Effects Of Environmental And Perceptual Uncertainty Using Deterministic Reinforcement Learning Dynamics With Partial Observability (2021)9.59
• Estimating Risk And Uncertainty In Deep Reinforcement Learning (2019)0.00
• Plan To Predict: Learning An Uncertainty-foreseeing Model For Model-based Reinforcement Learning (2023)0.00
• Learning Dynamics Model In Reinforcement Learning By Incorporating The Long Term Future (2019)0.00
• Dream To Explore: Adaptive Simulations For Autonomous Systems (2021)0.00
• Query The Agent: Improving Sample Efficiency Through Epistemic Uncertainty Estimation (2022)0.00
• Imagine-then-plan: Agent Learning From Adaptive Lookahead With World Models (2026)0.00
• How To Stay Curious While Avoiding Noisy Tvs Using Aleatoric Uncertainty Estimation (2021)0.00