Why structured outputs matter
The single biggest source of production LLM bugs is 'the model returned invalid JSON'. Every retry, every parse-and-fallback path, every silent failure that corrupts downstream state — most of it disappears when the model can't return invalid JSON in the first place.
Structured outputs move validation from your code into the model's decoding step. Instead of hoping the JSON parses, you constrain the tokens the model can emit to only those consistent with your schema. It's the difference between defensive engineering and correct-by-construction.
OpenAI: strict mode
Set response_format to {type: 'json_schema', json_schema: {schema, strict: true}}. On GPT-4o and newer, this guarantees the output matches the schema exactly — no missing fields, no extra keys, no wrong types.
Strict mode has a subset limitation: no oneOf at the top level, no minLength/maxLength on strings, all fields required by default. Design your schema around these from the start.
For tool calls, pass strict:true in the tool definition. Combined with tool_choice: 'required', you get guaranteed valid tool calls when a tool must fire.
Anthropic: tool_use blocks
Claude doesn't have a strict JSON mode. Instead, define a tool with an input_schema; Claude emits a tool_use block whose input field matches the schema in ~99% of cases without any additional validation.
For pure structured output (no side effect), define a tool named 'submit_response' whose only job is to receive the structured data. It's a lightweight pattern that gets you close to OpenAI's strict guarantee.
The remaining 1% failure is usually a missing required field on complex nested schemas. A single retry with the error message inline recovers cleanly.
Open-weights models
Llama 3.3, Mistral Small, and DeepSeek all support tool calling — with reliability that improves with each release but still trails frontier models. Add a validation layer with a single retry regardless.
For structured outputs at scale on open-weights models, use a constrained decoder: vLLM's guided generation, Outlines, or LMFormatEnforcer. These guarantee schema compliance at inference time and eliminate parse errors entirely.
Tradeoff: constrained decoding is slower per token and can degrade quality if the schema over-constrains reasoning. Reserve it for the outermost tool call, not internal chain-of-thought.
Patterns for production
Always validate the output even with strict mode enabled. Structured outputs guarantee schema shape; they don't guarantee semantic correctness. A number that's within schema but 100x too large is still a bug.
Log tool call inputs and validation errors separately from prose outputs. When something breaks, you want to know instantly whether it's a schema drift or a reasoning error.
Prefer flat schemas. Deeply nested objects increase both hallucination rate on open-weights models and the failure surface for schema drift over time.