Time-aware Q-networks: Resolving Temporal Irregularity For Deep Reinforcement Learning

Abstract

Deep Reinforcement Learning (DRL) has shown outstanding performance on inducing effective action policies that maximize expected long-term return on many complex tasks. Much of DRL work has been focused on sequences of events with discrete time steps and ignores the irregular time intervals between consecutive events. Given that in many real-world domains, data often consists of temporal sequences with irregular time intervals, and it is important to consider the time intervals between temporal events to capture latent progressive patterns of states. In this work, we present a general Time-Aware RL framework: Time-aware Q-Networks (TQN), which takes into account physical time intervals within a deep RL framework. TQN deals with time irregularity from two aspects: 1) elapsed time in the past and an expected next observation time for time-aware state approximation, and 2) action time window for the future for time-aware discounting of rewards. Experimental results show that by capturing

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykim2021time

Related papers

• Temporal Difference Models: Model-free Deep RL For Model-based Control (2018)0.00
• Training A Resilient Q-network Against Observational Interference (2021)0.00
• Simplifying Deep Temporal Difference Learning (2024)0.00
• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Adaptive \(q\)-network: On-the-fly Target Selection For Deep Reinforcement Learning (2024)0.00
• Discerning Temporal Difference Learning (2023)0.00
• On Improving Deep Reinforcement Learning For Pomdps (2017)0.00
• Adaptive Action Duration With Contextual Bandits For Deep Reinforcement Learning In Dynamic Environments (2025)0.00