projections
Every matrix multiply of the model: weights loaded in the format they are stored in, decode rows through a GEMV and prompt rows through a GEMM, each row's bits the same alone and batched.
required
In crit's library
Dense projectionsRows times dense matrices, the plain projection every quantized plugin is tested against.projectionsBit-exact expert rows on the CPUThe CPU computes 4-bit NVFP4 expert rows in the exact operation order the GPU kernel declares, so an expert gives the same bits wherever it runs.projectionsLow-bit weight projectionsWeights are stored as low-bit codes with one scale per group of values, and the projection multiplies the codes directly, applying each scale once, so decoding reads far fewer bytes.projections
Interface
The methods the engine calls on the slot's plugin, in their host form: token ids, rows of numbers and dicts. Every plugin of the slot has them, which is what lets an ablation switch one for another.
| Method | What it does |
|---|---|
prepare(...) | A matrix in its stored format |
project(...) | Rows times a matrix |
dense(...) | The dense reference product |
Device form: load(tensors, format, role) and matmul(x, matrix), each row's bits those of the row alone in a call of up to 8 rows
crit-hub