Continuous Deep Q-learning With Simulator For Stabilization Of Uncertain Discrete-time Systems

Abstract

Applications of reinforcement learning (RL) to stabilization problems of real systems are restricted since an agent needs many experiences to learn an optimal policy and may determine dangerous actions during its exploration. If we know a mathematical model of a real system, a simulator is useful because it predicates behaviors of the real system using the mathematical model with a given system parameter vector. We can collect many experiences more efficiently than interactions with the real system. However, it is difficult to identify the system parameter vector accurately. If we have an identification error, experiences obtained by the simulator may degrade the performance of the learned policy. Thus, we propose a practical RL algorithm that consists of two stages. At the first stage, we choose multiple system parameter vectors. Then, we have a mathematical model for each system parameter vector, which is called a virtual system. We obtain optimal Q-functions for multiple virtual sys

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyikemoto2021continuous

Related papers

• Reinforcement Learning With Fast Stabilization In Linear Dynamical Systems (2020)0.00
• Learning To Stabilize Online Reinforcement Learning In Unbounded State Spaces (2023)0.00
• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Provable Sim-to-real Transfer In Continuous Domain With Partial Observations (2022)0.00
• Stabilising Experience Replay For Deep Multi-agent Reinforcement Learning (2017)0.00
• Overcoming The Sim-to-real Gap: Leveraging Simulation To Learn To Explore For Real-world RL (2024)5.84
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89
• Automaton Distillation: Neuro-symbolic Transfer Learning For Deep Reinforcement Learning (2023)0.00