Structured outputs, and what to do when the model ignores them
Priya Raman ·
Constrained decoding guarantees you get valid JSON. It does not guarantee the values mean anything, and the failures get quieter as a result.
What we changed
Validate semantics, not just shape — an enum with a plausible wrong value passes every schema check.
Record the raw completion alongside the parsed object, or a parse failure is undebuggable.
Alert on a rising rate of fallback values. That is the signal the model is struggling.
The full walkthrough is in the documentation.