model
The architecture: its layer plan, its weight names and the forward pass, written over the other slots' plugins. Its object is the step model Core's concurrent runtime drives, so it computes prompt and decode rows of many requests in one step. A model plugin is the engine's own: crit's library holds none, since one family's layer arithmetic is no general mechanism.
required
In crit's library
No library plugin: each engine writes its own model plugin, since one family's layer arithmetic is no general mechanism.
Interface
The methods the engine calls on the slot's plugin, in their host form: token ids, rows of numbers and dicts. Every plugin of the slot has them, which is what lets an ablation switch one for another.
| Method | What it does |
|---|---|
info() | Core's settings and the engine's facts: vocabulary, context, max_concurrency of 2 or more, prefill chunk |
open(request) | A session for a request: prompt, output limit, sampling, stop tokens |
step(rows) | One result per scheduled row: prompt chunks and decode rows of many requests in one step |
commit(decisions) | Core's decision per row of the last step: tokens kept, an end, a cancel |
retire(session, finish) | The request ended; its state goes |
score(ids, ...) | Fixed text scored alone on the engine |
crit-hub