Metrics And Continuity In Reinforcement Learning

Abstract

In most practical applications of reinforcement learning, it is untenable to maintain direct estimates for individual states; in continuous-state systems, it is impossible. Instead, researchers often leverage state similarity (whether explicitly or implicitly) to build models that can generalize well from a limited set of samples. The notion of state similarity used, and the neighbourhoods and topologies they induce, is thus of crucial importance, as it will directly affect the performance of the algorithms. Indeed, a number of recent works introduce algorithms assuming the existence of "well-behaved" neighbourhoods, but leave the full specification of such topologies for future work. In this paper we introduce a unified formalism for defining these topologies through the lens of metrics. We establish a hierarchy amongst these metrics and demonstrate their theoretical implications on the Markov Decision Process specifying the reinforcement learning problem. We complement our theoretica

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylan2021metrics

Related papers

• Understanding Behavioral Metric Learning: A Large-scale Study On Distracting Reinforcement Learning Environments (2025)0.00
• Distributionally Robust Model-based Reinforcement Learning With Large State Spaces (2023)0.00
• Taming "data-hungry" Reinforcement Learning? Stability In Continuous State-action Spaces (2024)2.26
• Topological Foundations Of Reinforcement Learning (2024)0.00
• Review Of Metrics To Measure The Stability, Robustness And Resilience Of Reinforcement Learning (2022)0.00
• Towards Robust Bisimulation Metric Learning (2021)0.00
• A Kernel Perspective On Behavioural Metrics For Markov Decision Processes (2023)0.00
• On The Geometry Of Reinforcement Learning In Continuous State And Action Spaces (2022)0.00