Formal Ethical Obligations In Reinforcement Learning Agents: Verification And Policy Updates

Abstract

When designing agents for operation in uncertain environments, designers need tools to automatically reason about what agents ought to do, how that conflicts with what is actually happening, and how a policy might be modified to remove the conflict. These obligations include ethical and social obligations, permissions and prohibitions, which constrain how the agent achieves its mission and executes its policy. We propose a new deontic logic, Expected Act Utilitarian deontic logic, for enabling this reasoning at design time: for specifying and verifying the agent's strategic obligations, then modifying its policy from a reference policy to meet those obligations. Unlike approaches that work at the reward level, working at the logical level increases the transparency of the trade-offs. We introduce two algorithms: one for model-checking whether an RL agent has the right strategic obligations, and one for modifying a reference decision policy to make it meet obligations expressed in our l

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keysheablymyer2024formal

Related papers

• A Low-cost Ethics Shaping Approach For Designing Reinforcement Learning Agents (2017)0.00
• Online Learning Of Deceptive Policies Under Intermittent Observation (2025)0.00
• Toward Virtuous Reinforcement Learning: A Critique And Roadmap (2025)0.00
• Interpretable Multi-objective Reinforcement Learning Through Policy Orchestration (2018)0.00
• On Assessing The Safety Of Reinforcement Learning Algorithms Using Formal Methods (2021)0.00
• A Regulation Enforcement Solution For Multi-agent Reinforcement Learning (2019)2.26
• Structural Enforcement Of Goal Integrity In AI Agents Via Separation-of-powers Architecture (2026)0.00
• An Abstraction-based Method To Check Multi-agent Deep Reinforcement-learning Behaviors (2021)2.26