Experimental Results : Reinforcement Learning Of Pomdps Using Spectral Methods

Abstract

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging since the learner interacts with the environment and possibly changes the future observations in the process. We devise a learning algorithm running through epochs, in each epoch we employ spectral techniques to learn the POMDP parameters from a trajectory generated by a fixed policy. At the end of the epoch, an optimization oracle returns the optimal memoryless planning policy which maximizes the expected reward based on the estimated POMDP model. We prove an order-optimal regret bound with respect to the optimal memoryless policy and efficient scaling with respect to the dimensionality of observation and action spaces.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyazizzadenesheli2017experimental

Related papers

• Efficient Learning Of Pomdps With Known Observation Model In Average-reward Setting (2024)0.00
• A Spectral Approach To Off-policy Evaluation For Pomdps (2021)0.00
• Reinforcement Learning In Rich-observation Mdps Using Spectral Methods (2016)0.00
• Posterior Sampling-based Online Learning For Episodic Pomdps (2023)0.00
• Robust Reinforcement Learning In Pomdps With Incomplete And Noisy Observations (2019)0.00
• Provable Representation With Efficient Planning For Partial Observable Reinforcement Learning (2023)0.00
• Near-optimal Partially Observable Reinforcement Learning With Partial Online State Information (2023)0.00
• Sample-efficient Learning Of Pomdps With Multiple Observations In Hindsight (2023)0.00