Scalable Semantic Non-markovian Simulation Proxy For Reinforcement Learning

Abstract

Recent advances in reinforcement learning (RL) have shown much promise across a variety of applications. However, issues such as scalability, explainability, and Markovian assumptions limit its applicability in certain domains. We observe that many of these shortcomings emanate from the simulator as opposed to the RL training algorithms themselves. As such, we propose a semantic proxy for simulation based on a temporal extension to annotated logic. In comparison with two high-fidelity simulators, we show up to three orders of magnitude speed-up while preserving the quality of policy learned. In addition, we show the ability to model and leverage non-Markovian dynamics and instantaneous actions while providing an explainable trace describing the outcomes of the agent actions.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymukherji2023scalable

Related papers

• Influence-augmented Local Simulators: A Scalable Solution For Fast Deep RL In Large Networked Systems (2022)0.00
• Overcoming The Sim-to-real Gap: Leveraging Simulation To Learn To Explore For Real-world RL (2024)5.84
• Persim: Data-efficient Offline Reinforcement Learning With Heterogeneous Agents Via Personalized Simulators (2021)0.00
• Towards Data-driven Offline Simulations For Online Reinforcement Learning (2022)0.00
• Learning Symbolic Representations For Reinforcement Learning Of Non-markovian Behavior (2023)0.00
• Generalization Across Observation Shifts In Reinforcement Learning (2023)0.00
• Offsim: Offline Simulator For Model-based Offline Inverse Reinforcement Learning (2025)0.00
• S-REINFORCE: A Neuro-symbolic Policy Gradient Approach For Interpretable Reinforcement Learning (2023)0.00