Upside-down Reinforcement Learning For More Interpretable Optimal Control

Abstract

Model-Free Reinforcement Learning (RL) algorithms either learn how to map states to expected rewards or search for policies that can maximize a certain performance function. Model-Based algorithms instead, aim to learn an approximation of the underlying model of the RL environment and then use it in combination with planning algorithms. Upside-Down Reinforcement Learning (UDRL) is a novel learning paradigm that aims to learn how to predict actions from states and desired commands. This task is formulated as a Supervised Learning problem and has successfully been tackled by Neural Networks (NNs). In this paper, we investigate whether function approximation algorithms other than NNs can also be used within a UDRL framework. Our experiments, performed over several popular optimal control benchmarks, show that tree-based methods like Random Forests and Extremely Randomized Trees can perform just as well as NNs with the significant benefit of resulting in policies that are inherently more i

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycardenascartagena2024upside

Related papers

• All You Need Is Supervised Learning: From Imitation Learning To Meta-rl With Upside Down RL (2022)0.00
• Upside-down Reinforcement Learning Can Diverge In Stochastic Environments With Episodic Resets (2022)0.00
• Toward Interpretable Deep Reinforcement Learning With Linear Model U-trees (2018)13.05
• Learning Relative Return Policies With Upside-down Reinforcement Learning (2022)0.00
• Efficient Model-based Reinforcement Learning Through Optimistic Policy Search And Planning (2020)0.00
• Direct And Indirect Reinforcement Learning (2019)10.74
• Mitigating Information Loss In Tree-based Reinforcement Learning Via Direct Optimization (2024)0.00
• Barc: Backward Reachability Curriculum For Robotic Reinforcement Learning (2018)10.74