Missingness As Stability: Understanding The Structure Of Missingness In Longitudinal EHR Data And Its Impact On Reinforcement Learning In Healthcare

Abstract

There is an emerging trend in the reinforcement learning for healthcare literature. In order to prepare longitudinal, irregularly sampled, clinical datasets for reinforcement learning algorithms, many researchers will resample the time series data to short, regular intervals and use last-observation-carried-forward (LOCF) imputation to fill in these gaps. Typically, they will not maintain any explicit information about which values were imputed. In this work, we (1) call attention to this practice and discuss its potential implications; (2) propose an alternative representation of the patient state that addresses some of these issues; and (3) demonstrate in a novel but representative clinical dataset that our alternative representation yields consistently better results for achieving optimal control, as measured by off-policy policy evaluation, compared to representations that do not incorporate missingness information.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfleming2019missingness

Related papers

• An Empirical Study Of Representation Learning For Reinforcement Learning In Healthcare (2020)0.00
• Sample-efficient Reinforcement Learning Via Counterfactual-based Data Augmentation (2020)0.00
• Reinforcement Learning In Dynamic Treatment Regimes Needs Critical Reexamination (2024)2.35
• Semi-supervised Off Policy Reinforcement Learning (2020)0.00
• Clinician-in-the-loop Decision Making: Reinforcement Learning With Near-optimal Set-valued Policies (2020)0.00
• Counterfactually Fair Reinforcement Learning Via Sequential Data Preprocessing (2025)0.00
• Leveraging Factored Action Spaces For Efficient Offline Reinforcement Learning In Healthcare (2023)2.26
• A Relative Ignorability Framework For Decision-relevant Observability In Control Theory And Reinforcement Learning (2025)0.00