Task-optimal Exploration In Linear Dynamical Systems

Abstract

Exploration in unknown environments is a fundamental problem in reinforcement learning and control. In this work, we study task-guided exploration and determine what precisely an agent must learn about their environment in order to complete a particular task. Formally, we study a broad class of decision-making problems in the setting of linear dynamical systems, a class that includes the linear quadratic regulator problem. We provide instance- and task-dependent lower bounds which explicitly quantify the difficulty of completing a task of interest. Motivated by our lower bound, we propose a computationally efficient experiment-design based exploration algorithm. We show that it optimally explores the environment, collecting precisely the information needed to complete the task, and provide finite-time bounds guaranteeing that it achieves the instance- and task-optimal sample complexity, up to constant factors. Through several examples of the LQR problem, we show that performing task-gu

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywagenmaker2021task

Related papers

• Optimal Exploration For Model-based RL In Nonlinear Systems (2023)0.00
• An Optimal Policy For Learning Controllable Dynamics By Exploration (2025)0.00
• Optimistic Active Exploration Of Dynamical Systems (2023)0.00
• Sample-efficient Reinforcement Learning With Temporal Logic Objectives: Leveraging The Task Specification To Guide Exploration (2024)0.00
• Exploration Versus Exploitation In Reinforcement Learning: A Stochastic Control Approach (2018)9.76
• Dynamic Subgoal-based Exploration Via Bayesian Optimization (2019)0.00
• Directed Exploration In Reinforcement Learning From Linear Temporal Logic (2024)0.00
• Context-dependent Upper-confidence Bounds For Directed Exploration (2018)0.00