Policy Prediction Network: Model-free Behavior Policy With Model-based Learning In Continuous Action Space

Abstract

This paper proposes a novel deep reinforcement learning architecture that was inspired by previous tree structured architectures which were only useable in discrete action spaces. Policy Prediction Network offers a way to improve sample complexity and performance on continuous control problems in exchange for extra computation at training time but at no cost in computation at rollout time. Our approach integrates a mix between model-free and model-based reinforcement learning. Policy Prediction Network is the first to introduce implicit model-based learning to Policy Gradient algorithms for continuous action space and is made possible via the empirically justified clipping scheme. Our experiments are focused on the MuJoCo environments so that they can be compared with similar work done in this area.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywellmer2019policy

Related papers

• Discretizing Continuous Action Space With Unimodal Probability Distributions For On-policy Reinforcement Learning (2024)0.00
• Unified Policy Optimization For Continuous-action Reinforcement Learning In Non-stationary Tasks And Games (2022)2.26
• Specialized Deep Residual Policy Safe Reinforcement Learning-based Controller For Complex And Continuous State-action Spaces (2023)4.52
• Categorical Policies: Multimodal Policy Learning And Exploration In Continuous Control (2025)0.00
• On The Expressivity Of Neural Networks For Deep Reinforcement Learning (2019)0.00
• Exploring Reinforcement Learning Techniques For Discrete And Continuous Control Tasks In The Mujoco Environment (2023)0.00
• Policy-aware Model Learning For Policy Gradient Methods (2020)0.00
• On The Geometry Of Reinforcement Learning In Continuous State And Action Spaces (2022)0.00