Replicable Reinforcement Learning With Linear Function Approximation

Abstract

Replication of experimental results has been a challenge faced by many scientific disciplines, including the field of machine learning. Recent work on the theory of machine learning has formalized replicability as the demand that an algorithm produce identical outcomes when executed twice on different samples from the same distribution. Provably replicable algorithms are especially interesting for reinforcement learning (RL), where algorithms are known to be unstable in practice. While replicable algorithms exist for tabular RL settings, extending these guarantees to more practical function approximation settings has remained an open problem. In this work, we make progress by developing replicable methods for linear function approximation in RL. We first introduce two efficient algorithms for replicable random design regression and uncentered covariance estimation, each of independent interest. We then leverage these tools to provide the first provably efficient replicable RL algorithm

Stats

• citations4
• S2 citations—
• github stars0
• HF likes0
• heat score5.24
• arxiv keyeaton2025replicable

Related papers

• Provably Efficient Reinforcement Learning With Linear Function Approximation (2019)11.76
• Replicable Reinforcement Learning (2023)0.00
• Replicability In Reinforcement Learning (2023)0.00
• From Generative To Episodic: Sample-efficient Replicable Reinforcement Learning (2025)0.00
• Distributional Reinforcement Learning With Linear Function Approximation (2019)0.00
• List Replicable Reinforcement Learning (2025)0.00
• Distributionally Robust Offline Reinforcement Learning With Linear Function Approximation (2022)0.00
• Uniform-pac Bounds For Reinforcement Learning With Linear Function Approximation (2021)0.00