Using Logical Specifications Of Objectives In Multi-objective Reinforcement Learning

Abstract

It is notoriously difficult to control the behavior of reinforcement learning agents. Agents often learn to exploit the environment or reward signal and need to be retrained multiple times. The multi-objective reinforcement learning (MORL) framework separates a reward function into several objectives. An ideal MORL agent learns to generalize to novel combinations of objectives allowing for better control of an agent's behavior without requiring retraining. Many MORL approaches use a weight vector to parameterize the importance of each objective. However, this approach suffers from lack of expressiveness and interpretability. We propose using propositional logic to specify the importance of multiple objectives. By using a logic where predicates correspond directly to objectives, specifications are inherently more interpretable. Additionally the set of specifications that can be expressed with formal languages is a superset of what can be expressed by weight vectors. In this paper, we de

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynottingham2019using

Related papers

• Interpretability By Design For Efficient Multi-objective Reinforcement Learning (2025)0.00
• On Generalization Across Environments In Multi-objective Reinforcement Learning (2025)0.00
• Issues With Value-based Multi-objective Reinforcement Learning: Value Function Interference And Overestimation Sensitivity (2024)0.00
• Navigating Trade-offs: Policy Summarization For Multi-objective Reinforcement Learning (2024)2.26
• Provable Multi-objective Reinforcement Learning With Generative Models (2020)0.00
• Multi-objective Reinforcement Learning Based On Decomposition: A Taxonomy And Framework (2023)9.92
• Toward Negotiable Reinforcement Learning: Shifting Priorities In Pareto Optimal Sequential Decision-making (2017)0.00
• A Generalized Algorithm For Multi-objective Reinforcement Learning And Policy Adaptation (2019)0.00