Counterfactually Fair Reinforcement Learning Via Sequential Data Preprocessing

Abstract

When applied in healthcare, reinforcement learning (RL) seeks to dynamically match the right interventions to subjects to maximize population benefit. However, the learned policy may disproportionately allocate efficacious actions to one subpopulation, creating or exacerbating disparities in other socioeconomically-disadvantaged subgroups. These biases tend to occur in multi-stage decision making and can be self-perpetuating, which if unaccounted for could cause serious unintended consequences that limit access to care or treatment benefit. Counterfactual fairness (CF) offers a promising statistical tool grounded in causal inference to formulate and study fairness. In this paper, we propose a general framework for fair sequential decision making. We theoretically characterize the optimal CF policy and prove its stationarity, which greatly simplifies the search for optimal CF policies by leveraging existing RL algorithms. The theory also motivates a sequential data preprocessing algorit

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2025counterfactually

Related papers

• Sample-efficient Reinforcement Learning Via Counterfactual-based Data Augmentation (2020)0.00
• Learning Impartial Policies For Sequential Counterfactual Explanations Using Deep Reinforcement Learning (2023)0.00
• What Hides Behind Unfairness? Exploring Dynamics Fairness In Reinforcement Learning (2024)0.95
• Clinician-in-the-loop Decision Making: Reinforcement Learning With Near-optimal Set-valued Policies (2020)0.00
• Striking A Balance In Fairness For Dynamic Systems Through Reinforcement Learning (2024)2.26
• Leveraging Factored Action Spaces For Efficient Offline Reinforcement Learning In Healthcare (2023)2.26
• Achieving Fairness In Multi-agent Markov Decision Processes Using Reinforcement Learning (2023)0.00
• Fairness In Reinforcement Learning (2016)0.00