crit-hub

attention

Each token's queries over its sequence's keys and values, up to its position, read from the KV cache's pages.

required

In crit's library

Interface

The methods the engine calls on the slot's plugin, in their host form: token ids, rows of numbers and dicts. Every plugin of the slot has them, which is what lets an ablation switch one for another.

MethodWhat it does
attend(...)Each row's queries over its sequence's keys and values

Device form: forward(layer, queries, pool, batch, scale): each token attends its sequence's KV up to its position