Ecological Reinforcement Learning

Abstract

Much of the current work on reinforcement learning studies episodic settings, where the agent is reset between trials to an initial state distribution, often with well-shaped reward functions. Non-episodic settings, where the agent must learn through continuous interaction with the world without resets, and where the agent receives only delayed and sparse reward signals, is substantially more difficult, but arguably more realistic considering real-world environments do not present the learner with a convenient "reset mechanism" and easy reward shaping. In this paper, instead of studying algorithmic improvements that can address such non-episodic and sparse reward settings, we instead study the kinds of environment properties that can make learning under such conditions easier. Understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable. We first discuss what we term "envi

Stats

• citations12
• S2 citations—
• github stars0
• HF likes0
• heat score8.35
• arxiv keycoreyes2020ecological

Related papers

• Improving Generalization To New Environments And Removing Catastrophic Forgetting In Reinforcement Learning By Using An Eco-system Of Agents (2022)0.00
• Discovering Minimal Reinforcement Learning Environments (2024)0.00
• Environment Shaping In Reinforcement Learning Using State Abstraction (2020)0.00
• Learning To Design Games: Strategic Environments In Reinforcement Learning (2017)0.00
• Episodic Reinforcement Learning With Expanded State-reward Space (2024)0.00
• Experiential Reinforcement Learning (2026)2.58
• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Online Reinforcement Learning In Non-stationary Context-driven Environments (2023)0.00