Quantifying The Impact Of Non-stationarity In Reinforcement Learning-based Traffic Signal Control

Abstract

In reinforcement learning (RL), dealing with non-stationarity is a challenging issue. However, some domains such as traffic optimization are inherently non-stationary. Causes for and effects of this are manifold. In particular, when dealing with traffic signal controls, addressing non-stationarity is key since traffic conditions change over time and as a function of traffic control decisions taken in other parts of a network. In this paper we analyze the effects that different sources of non-stationarity have in a network of traffic signals, in which each signal is modeled as a learning agent. More precisely, we study both the effects of changing the \textit\{context\} in which an agent learns (e.g., a change in flow rates experienced by it), as well as the effects of reducing agent observability of the true environment state. Partial observability may cause distinct states (in which distinct actions are optimal) to be seen as the same by the traffic signal agents. This, in turn, may l

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keyalegre2020quantifying

Related papers

• Testing Stationarity And Change Point Detection In Reinforcement Learning (2022)0.00
• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Online Reinforcement Learning In Non-stationary Context-driven Environments (2023)0.00
• Safe Continual Reinforcement Learning In Non-stationary Environments (2026)12.89
• Model-agnostic Solutions For Deep Reinforcement Learning In Non-ergodic Contexts (2026)0.00
• Reinforcement Learning For Datacenter Congestion Control (2021)0.00
• A Survey Of Learning In Multiagent Environments: Dealing With Non-stationarity (2017)0.00
• Reinforcement Learning With Non-ergodic Reward Increments: Robustness Via Ergodicity Transformations (2023)0.00