Unifying Task Specification In Reinforcement Learning

Abstract

Reinforcement learning tasks are typically specified as Markov decision processes. This formalism has been highly successful, though specifications often couple the dynamics of the environment and the learning objective. This lack of modularity can complicate generalization of the task specification, as well as obfuscate connections between different task settings, such as episodic and continuing. In this work, we introduce the RL task formalism, that provides a unification through simple constructs including a generalization to transition-based discounting. Through a series of examples, we demonstrate the generality and utility of this formalism. Finally, we extend standard learning constructs, including Bellman operators, and extend some seminal theoretical results, including approximation errors bounds. Overall, we provide a well-understood and sound formalism on which to build theoretical results and simplify algorithm use and development.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywhite2016unifying

Related papers

• Learning Task Automata For Reinforcement Learning Using Hidden Markov Models (2022)2.26
• Logical Specifications-guided Dynamic Task Sampling For Reinforcement Learning Agents (2024)2.26
• Extended Markov Games To Learn Multiple Tasks In Multi-agent Reinforcement Learning (2020)3.58
• Verifiable And Compositional Reinforcement Learning Systems (2021)0.00
• The Impact Of Task Underspecification In Evaluating Deep Reinforcement Learning (2022)4.52
• On The Model-misspecification In Reinforcement Learning (2023)0.00
• Adaptive Reward Design For Reinforcement Learning (2024)0.00
• Sample-efficient Reinforcement Learning With Temporal Logic Objectives: Leveraging The Task Specification To Guide Exploration (2024)0.00