Loading…
Glossary term
A speedup technique in which a small model drafts tokens and a larger model checks them.
A small, fast draft model proposes several next tokens at once. The large model then checks all of them in a single pass, accepting those it agrees with and fixing the first one it does not.
The final text is the same as the large model would have produced, only faster.
It speeds up generation without changing quality, cutting latency and cost.