← all papers · overview

When Drafts Evolve: Speculative Decoding Meets Online Learning

Abstract

Speculative decoding has emerged as a widely adopted paradigm for accelerating large language model inference, where a lightweight draft model rapidly generates candidate tokens that are then verified in parallel by a larger target model. However, due to limited model capacity, drafts often struggle to approximate the target distribution, resulting in shorter acceptance lengths and diminished spee

Related papers

Ranked by semantic similarity — how closely each paper's abstract matches this one (100% = near-identical topic).