Offline Behavioral Data Selection

Abstract

Behavioral cloning is a widely adopted approach for offline policy learning from expert demonstrations. However, the large scale of offline behavioral datasets often results in computationally intensive training when used in downstream tasks. In this paper, we uncover the striking data saturation in offline behavioral data: policy performance rapidly saturates when trained on a small fraction of the dataset. We attribute this effect to the weak alignment between policy performance and test loss, revealing substantial room for improvement through data selection. To this end, we propose a simple yet effective method, Stepwise Dual Ranking (SDR), which extracts a compact yet informative subset from large-scale offline behavioral datasets. SDR is build on two key principles: (1) stepwise clip, which prioritizes early-stage data; and (2) dual ranking, which selects samples with both high action-value rank and low state-density rank. Extensive experiments and ablation studies on D4RL benchma

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keylei2025offline

Related papers

• Know Your Boundaries: The Necessity Of Explicit Behavioral Cloning In Offline RL (2022)0.00
• When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning? (2022)0.00
• Behavior Estimation From Multi-source Data For Offline Reinforcement Learning (2022)2.26
• Beyond Uniform Sampling: Offline Reinforcement Learning With Imbalanced Datasets (2023)2.83
• A Dataset Perspective On Offline Reinforcement Learning (2021)0.00
• Expert Or Not? Assessing Data Quality In Offline Reinforcement Learning (2025)0.00
• Adaptive Behavior Cloning Regularization For Stable Offline-to-online Reinforcement Learning (2022)8.09
• Data Valuation For Offline Reinforcement Learning (2022)0.00