Improving Human Performance With Value-aware Interventions: A Case Study In Chess

Abstract

AI systems are increasingly used to assist humans in sequential decision-making tasks, yet determining when and how an AI assistant should intervene remains a fundamental challenge. A potential baseline is to recommend the optimal action according to a strong model. However, such actions assume optimal follow-up actions, which human decision makers may fail to execute, potentially reducing overall performance. In this work, we propose and study value-aware interventions, motivated by a basic principle in reinforcement learning: under the Bellman equation, the optimal policy selects actions that maximize the immediate reward plus the value function. When a decision maker follows a suboptimal policy, this policy-value consistency no longer holds, creating discrepancies between the actions taken by the policy and those that maximize the immediate reward plus the value of the next state. We show that these policy-value inconsistencies naturally identify opportunities for intervention. We f

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keynarayanan2026improving

Related papers

• Learning To Make Adherence-aware Advice (2023)0.00
• Towards Optimizing Human-centric Objectives In Ai-assisted Decision-making With Offline Reinforcement Learning (2024)0.00
• Policy-value Alignment And Robustness In Search-based Multi-agent Learning (2023)0.00
• In Pursuit Of Predictive Models Of Human Preferences Toward AI Teammates (2025)0.00
• Blessing From Human-ai Interaction: Super Reinforcement Learning In Confounded Environments (2022)0.00
• Human-ai Learning Performance In Multi-armed Bandits (2018)7.50
• Enhancing Human Experience In Human-agent Collaboration: A Human-centered Modeling Approach Based On Positive Human Gain (2024)0.00
• Probe-based Interventions For Modifying Agent Behavior (2022)0.00