constrain
Core's structured-output service keeps a reply inside a JSON schema, a regex, a list of choices or a grammar.
crit Core
What happens here
- It compiles the request's constraint once, when the request comes in.
- Each step it gives every constrained row the set of tokens that keep the reply valid.
- The sampler then picks only from that set.
Learn more
The sampler plugin that applies the set.
crit-hub