crit-hub

model

The architecture: its layer plan, its weight names and the forward pass, written over the other slots' plugins. Its object is the step model Core's concurrent runtime drives, so it computes prompt and decode rows of many requests in one step. A model plugin is the engine's own: crit's library holds none, since one family's layer arithmetic is no general mechanism.

required

In crit's library

No library plugin: each engine writes its own model plugin, since one family's layer arithmetic is no general mechanism.

Interface

The methods the engine calls on the slot's plugin, in their host form: token ids, rows of numbers and dicts. Every plugin of the slot has them, which is what lets an ablation switch one for another.

MethodWhat it does
info()Core's settings and the engine's facts: vocabulary, context, max_concurrency of 2 or more, prefill chunk
open(request)A session for a request: prompt, output limit, sampling, stop tokens
step(rows)One result per scheduled row: prompt chunks and decode rows of many requests in one step
commit(decisions)Core's decision per row of the last step: tokens kept, an end, a cancel
retire(session, finish)The request ended; its state goes
score(ids, ...)Fixed text scored alone on the engine