Abstract
Imitation learning offers an efficient framework for robotic skill acquisition. However, current methods struggle with accurate action-target association, limited generalization, and long-horizon tasks with sequential constraints. To address these, we propose Task-DP3, a goal-focused framework for sequential multiobject manipulation. It includes a goal-conditioned point cloud sampler that extracts target-centric point clouds from segmentation masks, and a perception-driven skill scheduler that dynamically determines task states and plans skill sequences. This enables adaptive trajectory generation in response to environmental changes. Real-robot experiments show Task-DP3 achieves a 92.5% success rate in multiobject tasks with only 30 demonstrations, outperforming state-of-the-art methods. It also demonstrates strong generalization to unseen clutter and backgrounds and excels in long-horizon tasks with strict order constraints, proving highly suitable for real-world diffusion-based imitation learning.