Opinion-guided Reinforcement Learning

Abstract

Human guidance is often desired in reinforcement learning to improve the performance of the learning agent. However, human insights are often mere opinions and educated guesses rather than well-formulated arguments. While opinions are subject to uncertainty, e.g., due to partial informedness or ignorance about a problem, they also emerge earlier than hard evidence can be produced. Thus, guiding reinforcement learning agents by way of opinions offers the potential for more performant learning processes, but comes with the challenge of modeling and managing opinions in a formal way. In this article, we present a method to guide reinforcement learning agents through opinions. To this end, we provide an end-to-end method to model and manage advisors' opinions. To assess the utility of the approach, we evaluate it with synthetic (oracle) and human advisors, at different levels of uncertainty, and under multiple advice strategies. Our results indicate that opinions, even if uncertain, improv

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keydagenais2024opinion

Related papers

• Reinforcement Learning With Human Advice: A Survey (2020)0.00
• Human Engagement Providing Evaluative And Informative Advice For Interactive Reinforcement Learning (2020)9.23
• Influencing Reinforcement Learning Through Natural Language Guidance (2021)0.00
• Directed Policy Gradient For Safe Reinforcement Learning With Human Advice (2018)0.00
• Pref-guide: Continual Policy Learning From Real-time Human Feedback Via Preference-based Learning (2025)0.00
• Implications Of Human Irrationality For Reinforcement Learning (2020)0.00
• Learning Shaping Strategies In Human-in-the-loop Interactive Reinforcement Learning (2018)0.00
• When Your Ais Deceive You: Challenges Of Partial Observability In Reinforcement Learning From Human Feedback (2024)0.00