Pomdps In Continuous Time And Discrete Spaces

Abstract

Many processes, such as discrete event systems in engineering or population dynamics in biology, evolve in discrete space and continuous time. We consider the problem of optimal decision making in such discrete state and action space systems under partial observability. This places our work at the intersection of optimal filtering and optimal control. At the current state of research, a mathematical description for simultaneous decision making and filtering in continuous time with finite state and action spaces is still missing. In this paper, we give a mathematical description of a continuous-time partial observable Markov decision process (POMDP). By leveraging optimal filtering theory we derive a Hamilton-Jacobi-Bellman (HJB) type equation that characterizes the optimal solution. Using techniques from deep learning we approximately solve the resulting partial integro-differential equation. We present (i) an approach solving the decision problem offline by learning an approximation o

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyalt2020pomdps

Related papers

• Sequential Monte Carlo For Policy Optimization In Continuous Pomdps (2025)0.00
• Sparse Tree Search Optimality Guarantees In Pomdps With Continuous Observation Spaces (2019)5.84
• Bayesian Learning Of Optimal Policies In Markov Decision Processes With Countably Infinite State-space (2023)0.00
• A Policy Gradient Method For Confounded Pomdps (2023)0.00
• Model-based Learning Of Near-optimal Finite-window Policies In Pomdps (2026)0.00
• Computationally Efficient PAC RL In Pomdps With Latent Determinism And Conditional Embeddings (2022)0.00
• Near-optimal Partially Observable Reinforcement Learning With Partial Online State Information (2023)0.00
• Bayes-cpace: PAC Optimal Exploration In Continuous Space Bayes-adaptive Markov Decision Processes (2018)0.00