crit-hub

constrain

Core's structured-output service keeps a reply inside a JSON schema, a regex, a list of choices or a grammar.

crit Core

What happens here

  1. It compiles the request's constraint once, when the request comes in.
  2. Each step it gives every constrained row the set of tokens that keep the reply valid.
  3. The sampler then picks only from that set.

Learn more

The sampler plugin that applies the set.