Sample-efficient Reinforcement Learning With Temporal Logic Objectives: Leveraging The Task Specification To Guide Exploration

Abstract

This paper addresses the problem of learning optimal control policies for systems with uncertain dynamics and high-level control objectives specified as Linear Temporal Logic (LTL) formulas. Uncertainty is considered in the workspace structure and the outcomes of control decisions giving rise to an unknown Markov Decision Process (MDP). Existing reinforcement learning (RL) algorithms for LTL tasks typically rely on exploring a product MDP state-space uniformly (using e.g., an \(\epsilon\)-greedy policy) compromising sample-efficiency. This issue becomes more pronounced as the rewards get sparser and the MDP size or the task complexity increase. In this paper, we propose an accelerated RL algorithm that can learn control policies significantly faster than competitive approaches. Its sample-efficiency relies on a novel task-driven exploration strategy that biases exploration towards directions that may contribute to task satisfaction. We provide theoretical analysis and extensive compara

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykantaros2024sample

Related papers

• Directed Exploration In Reinforcement Learning From Linear Temporal Logic (2024)0.00
• Sample Efficient Model-free Reinforcement Learning From LTL Specifications With Optimality Guarantees (2023)0.00
• Logical Specifications-guided Dynamic Task Sampling For Reinforcement Learning Agents (2024)2.26
• A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks (2017)11.58
• Learning Probabilistic Temporal Logic Specifications For Stochastic Systems (2025)0.00
• Regret-free Reinforcement Learning For LTL Specifications (2024)0.00
• On The (in)tractability Of Reinforcement Learning For LTL Objectives (2021)0.00
• Adaptive Reward Design For Reinforcement Learning (2024)0.00