.png){kind=icon}
Return
Block glancing for flash speculative decoding
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2026.135363.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The autoregressive decoding nature of large language models (LLMs) imposes significant inference latency, motivating speculative decoding as a promising acceleration paradigm. Recently, diffusion-based draft models such as DFlash have been proposed to generate draft tokens in parallel within a single forward pass, achieving notable speedups. However, DFlash assumes conditional independence among tokens within each block, neglecting the inherent sequential dependencies and syntactic constraints between contiguous tokens. This assumption leads to incoherent draft sequences, inaccurate semantic predictions, and high token rejection rates during verification, thereby limiting overall acceleration efficiency. To address this limitation, we propose a block glancing strategy that introduces a layer-wise residual glancing mechanism for training diffusion-based draft models. Specifically, our method progressively reveals ground-truth token embeddings at intermediate layers of the draft model, with the reveal ratio jointly determined by the layer depth and the layer-wise prediction accuracy, enabling the model to capture intra-block token dependencies while preserving parallel inference. Extensive experiments on Qwen3 models across three benchmark tasks demonstrate that our method achieves significant speedup improvements over current diffusion speculative decoding.
Keywords:
Large language models
Speculative decoding
Diffusion language models
Block glancing
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
Cited Papers
No cited papers available