crit-hub

Structure

Every crit engine is crit Core with plugins in its slots. Core's own nodes are the same in every engine. Each slot is a part that depends on the model or the GPU, and an engine puts one plugin in each slot it uses. How a crit engine is built

The slots and crit's library

NodeWhat happens hereFilled by
loadReads the engine's configuration.json, checks that its plugins fit together and puts each in its slotcrit Core
placementrequiredWhere each tensor lives: GPU, host memory or disk
requestA request comes in: the chat template, the prompt's tokens, admissioncrit Core
scheduleRows of many requests go into one stepcrit Core
graphsWhich captured CUDA graph a step runs
modelrequiredThe model's architecture: its layers, run over the slots after itthe engine's own
embeddingsToken ids to vectors
normsNormalization, rotary positions and gated products around the matmuls
projectionsrequiredEvery matrix multiply
attentionrequiredEach token looks back over the ones before it
kv_cacherequiredKeys and values kept for every request
recurrenceThe state of linear-attention layers, in hybrid models
routerPicks each token's experts, in mixture-of-experts models
expertsRuns the picked experts
expert_sourceWhere expert weights come from
expert_residencyWhich experts stay on the GPU
constrainStructured output: the tokens a row may take nextcrit Core
samplerrequiredPicks the next token
drafterGuesses the next few tokens ahead
verifierKeeps the guesses the model agrees with
streamStop rules, the reply streamed back, crit serve and crit-evalcrit Core