drafter
Proposes the draft tokens a speculative round verifies: a chain or a tree, from the model's draft head, the prompt or a published drafter.
optionalhost policy
In crit's library
MTP speculative decodingThe model's own multi-token prediction layer guesses the next few tokens, and one pass of the full model checks them all at once.drafterDraft treesThe drafter keeps its runner-up guesses as branches of a tree, one pass of the full model checks every branch, and the round keeps the longest path it agrees with.drafterverifierDraft vocabulary subsetThe drafter scores only the output-head rows of the tokens the model writes most, so drafts get cheaper, while verification keeps the full head and greedy output is unchanged.drafterHidden-state draftersA small published drafter reads hidden states from a few of the target model's layers and guesses the next tokens, one per step or a whole block per pass.drafterPrompt lookup decodingDrafts are copied from wherever the last few tokens appeared earlier in the prompt or output, and one pass of the full model checks them all.drafter
Interface
The methods the engine calls on the slot's plugin, in their host form: token ids, rows of numbers and dicts. Every plugin of the slot has them, which is what lets an ablation switch one for another.
| Method | What it does |
|---|---|
start(sequence, tokens) | A sequence's tokens so far |
propose(sequence, width) | The draft tree to verify: tokens, parents, laws |
observe(sequence, emitted) | What the round emitted |
release(sequence) | The sequence ended |
crit-hub