Reinforcement Learning Via Conservative Agent For Environments With Random Delays

Abstract

Real-world reinforcement learning applications are often hindered by delayed feedback from environments, which violates the Markov assumption and introduces significant challenges. Although numerous delay-compensating methods have been proposed for environments with constant delays, environments with random delays remain largely unexplored due to their inherent variability and unpredictability. In this study, we propose a simple yet robust agent for decision-making under random delays, termed the conservative agent, which reformulates the random-delay environment into its constant-delay equivalent. This transformation enables any state-of-the-art constant-delay method to be directly extended to the random-delay environments without modifying the algorithmic structure or sacrificing performance. We evaluate the conservative agent-based algorithm on continuous control tasks, and empirical results demonstrate that it significantly outperforms existing baseline algorithms in terms of asymp

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylee2025reinforcement

Related papers

• Reinforcement Learning With Random Delays (2020)0.00
• Delay-aware Multi-agent Reinforcement Learning For Cooperative And Competitive Environments (2020)0.00
• Model-based Reinforcement Learning Under Random Observation Delays (2025)0.00
• Revisiting State Augmentation Methods For Reinforcement Learning With Stochastic Delays (2021)10.35
• Conservative Exploration In Reinforcement Learning (2020)0.00
• Reinforcement Learning For Control Systems With Time Delays: A Comprehensive Survey (2026)0.00
• Simple Agent, Complex Environment: Efficient Reinforcement Learning With Agent States (2021)0.00
• Conservative Exploration For Policy Optimization Via Off-policy Policy Evaluation (2023)0.00