crit-hub

experts

The routed experts' SwiGLU blocks for each token row, wherever the experts are.

optional

In crit's library

Interface

The methods the engine calls on the slot's plugin, in their host form: token ids, rows of numbers and dicts. Every plugin of the slot has them, which is what lets an ablation switch one for another.

MethodWhat it does
run(...)The routed block of each token row

Device form: load(layer, expert, gate_up, down) and forward(layer, x, ids, weights)