Optimizing Algorithms For Mobile Health Interventions With Active Querying Optimization

Abstract

Reinforcement learning in mobile health (mHealth) interventions requires balancing intervention efficacy with user burden, particularly when state measurements (for example, user surveys or feedback) are costly yet essential. The Act-Then-Measure (ATM) heuristic addresses this challenge by decoupling control and measurement actions within the Action-Contingent Noiselessly Observable Markov Decision Process (ACNO-MDP) framework. However, the standard ATM algorithm relies on a temporal-difference-inspired Q-learning method, which is prone to instability in sparse and noisy environments. In this work, we propose a Bayesian extension to ATM that replaces standard Q-learning with a Kalman filter-style Bayesian update, maintaining uncertainty-aware estimates of Q-values and enabling more stable and sample-efficient learning. We evaluate our method in both toy environments and clinically motivated testbeds. In small, tabular environments, Bayesian ATM achieves comparable or improved scalarize

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyrawashdeh2025optimizing

Related papers

• Fast Physical Activity Suggestions: Efficient Hyperparameter Learning In Mobile Health (2020)0.00
• Act-then-measure: Reinforcement Learning For Partially Observable Environments With Active Measuring (2023)3.58
• IRL For Restless Multi-armed Bandits With Applications In Maternal And Child Health (2024)0.00
• Cohesion-based Online Actor-critic Reinforcement Learning For Mhealth Intervention (2017)5.24
• Optimizing The Long-term Average Reward For Continuing Mdps: A Technical Report (2021)0.00
• Alternating Optimisation And Quadrature For Robust Control (2016)7.16
• Effective Warm Start For The Online Actor-critic Reinforcement Learning Based Mhealth Intervention (2017)0.00
• Clinician-in-the-loop Decision Making: Reinforcement Learning With Near-optimal Set-valued Policies (2020)0.00