Structure
Every crit engine is crit Core with plugins in its slots. Core's own nodes are the same in every engine. Each slot is a part that depends on the model or the GPU, and an engine puts one plugin in each slot it uses. How a crit engine is built
The slots and crit's library
| Node | What happens here | Filled by |
|---|---|---|
| load | Reads the engine's configuration.json, checks that its plugins fit together and puts each in its slot | crit Core |
| placementrequired | Where each tensor lives: GPU, host memory or disk | |
| request | A request comes in: the chat template, the prompt's tokens, admission | crit Core |
| schedule | Rows of many requests go into one step | crit Core |
| graphs | Which captured CUDA graph a step runs | |
| modelrequired | The model's architecture: its layers, run over the slots after it | the engine's own |
| embeddings | Token ids to vectors | |
| norms | Normalization, rotary positions and gated products around the matmuls | |
| projectionsrequired | Every matrix multiply | |
| attentionrequired | Each token looks back over the ones before it | |
| kv_cacherequired | Keys and values kept for every request | |
| recurrence | The state of linear-attention layers, in hybrid models | |
| router | Picks each token's experts, in mixture-of-experts models | |
| experts | Runs the picked experts | |
| expert_source | Where expert weights come from | |
| expert_residency | Which experts stay on the GPU | |
| constrain | Structured output: the tokens a row may take next | crit Core |
| samplerrequired | Picks the next token | |
| drafter | Guesses the next few tokens ahead | |
| verifier | Keeps the guesses the model agrees with | |
| stream | Stop rules, the reply streamed back, crit serve and crit-eval | crit Core |
crit-hub