Deep Reinforcement Learning Based Networked Control With Network Delays For Signal Temporal Logic Specifications

Abstract

We apply deep reinforcement learning (DRL) to design of a networked controller with network delays to complete a temporal control task that is described by a signal temporal logic (STL) formula. STL is useful to deal with a specification with a bounded time interval for a dynamical system. In general, an agent needs not only the current system state but also the past behavior of the system to determine a desired control action for satisfying the given STL formula. Additionally, we need to consider the effect of network delays for data transmissions. Thus, we propose an extended Markov decision process using past system states and control actions, which is called a \(\tau d\)-MDP, so that the agent can evaluate the satisfaction of the STL formula considering the network delays. Thereafter, we apply a DRL algorithm to design a networked controller using the \(\tau d\)-MDP. Through simulations, we also demonstrate the learning performance of the proposed algorithm.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyikemoto2021deep

Related papers

• Funnel-based Reward Shaping For Signal Temporal Logic Tasks In Reinforcement Learning (2022)7.16
• Reinforcement Learning For Control Systems With Time Delays: A Comprehensive Survey (2026)0.00
• Revisiting State Augmentation Methods For Reinforcement Learning With Stochastic Delays (2021)10.35
• Temporal Difference Models: Model-free Deep RL For Model-based Control (2018)0.00
• A Hierarchical Reinforcement Learning Method For Persistent Time-sensitive Tasks (2016)0.00
• Regret-free Reinforcement Learning For LTL Specifications (2024)0.00
• Application Of Deep Reinforcement Learning To Event-triggered Control For Networked Artificial Pancreas Systems (2026)0.00
• Sample-efficient Reinforcement Learning With Temporal Logic Objectives: Leveraging The Task Specification To Guide Exploration (2024)0.00