Structured Outputs as Data Contracts: JSON Schema, Validation, Repair, and Retry

The 2 a.m. page is almost always the same story. A pipeline that extracted clean JSON in the demo starts throwing parse errors in production, and the logs contain nothing useful — just a truncated string, or a markdown fence wrapped around an object that almost parses. The model did not get worse. The demo just never tested the boundary.

Here is the mental model that fixes this: the schema is not an instruction you hope the model follows. It is a data contract, and the model is an untrusted producer on the other side of it. Every byte crossing that boundary gets validated before you trust it, and every rejection leaves evidence behind.

That reframe changes what you build. You stop tuning prompt wording and start engineering a gate.

Why Prompted JSON Is Not a Contract

There are three distinct reliability layers in any structured generation task, and conflating them is the root of most production failures.

Layer one: valid JSON syntax. The output parses. This is the weakest useful guarantee. A parseable object with a hallucinated customer ID passes every syntax check and still corrupts your database.

Layer two: schema conformance. The output matches the shape you declared — required fields present, types correct, enums respected. Constrained decoding can make this near-certain. Nothing makes it semantically correct.

Layer three: semantic correctness. The values are actually right. Does that order ID exist? Does end_date follow start_date? Is the extracted dollar amount plausible? No generation-time mechanism addresses this automatically.

The failure taxonomy is predictable once you name the layers:

Failure	Layer	Typical cause
Truncated output	Syntax	Token limit hit mid-object
Markdown fences, prose preamble	Syntax	Prompt did not forbid wrapping
Missing required field	Conformance	Model omitted rather than invented
Type drift (string where number expected)	Conformance	Ambiguous source text
Enum hallucination	Conformance	Value outside the closed set
Valid shape, wrong value	Semantic	Model guessed instead of extracting

The distinction between constrained decoding and post-hoc validation matters because they fail differently. Constrained decoding enforces grammar at generation time — the model cannot emit a token that violates the schema. Post-hoc validation checks what came back after the fact. You need both. Constrained decoding cannot catch a hallucinated ID, and post-hoc validation cannot catch a truncation that happened before the closing brace.

The schema is a contract. The prompt is a request. When they disagree, the contract wins, and the prompt is just documentation.

Acceptance criteria belong in the prompt. Enforcement belongs in code. If you have already written prompt contracts that specify objectives, inputs, and acceptance criteria, you have the specification half. This is the enforcement half.

Designing the Schema as an Interface

Most teams derive their schema from an existing database model or an internal API type. That is backwards. Start from the consumer: what does the downstream function actually need, and what can it tolerate being absent?

Required versus optional is a semantic decision, not a formatting one. Mark everything required and the model will invent values when the source text lacks them — a fabricated phone_number is worse than a null. Mark too much optional and every caller inherits null-handling logic. The right question is: if this field is missing, does the downstream code have a sensible default, or does it need to know?

Enums are the highest-leverage constraint you can add. They convert an open generation problem into a classification problem. A status field constrained to ["pending", "active", "closed"] eliminates an entire class of downstream branching bugs. Every free-form string field that could be an enum should be one.

Flatten aggressively. Deep nesting, recursive structures, unions, and free-form maps are where schema-constrained generation degrades and where validation errors become hard to attribute. A validation error on data.items[3].metadata.tags[0] tells you almost nothing. A flat schema with item_3_tag_0 is ugly but debuggable. Prefer shallow and wide over deep and narrow.

Field names and descriptions are in-band documentation. The model reads them. A field called amount with description "Total in USD, numeric only, no currency symbol" is part of the interface, not a comment. Treat descriptions as you would treat a function signature that a junior developer will read without context.

One subtlety that bites teams: the schema you send and the schema you enforce are not always the same object. Some providers simplify schemas before they reach the model — dropping numeric ranges, string length constraints, or format checks that their constrained decoder cannot express. The model receives a looser contract; your validator still enforces the original. This is fine, but only if you know it is happening. If you assume the model is enforcing minimum: 100 and your validator is not, you have a silent gap.

Choosing an Enforcement Mechanism

The decision axis is not "which feature is newest." It is: how much does a malformed response cost you, and how much schema complexity can the mechanism actually express?

Native structured output with constrained decoding gives the strongest syntactic guarantee. The provider compiles your schema into a grammar and constrains sampling. Schema expressiveness is capped — provider dialects differ, and some constraints get simplified before reaching the model. Behavior also varies by model version, so a schema that works today may need adjustment after a model upgrade.

Tool calling as a structured-output channel is useful when the same call must both act and return data. But it conflates two concerns: the action and the data contract. Retry semantics get complicated because a retry might re-fire the action. If you only need data back, use the structured output path directly.

Prompt-only JSON with a validator is the weakest guarantee but the only option for models or self-hosted runtimes without constrained decoding. It is also the right fallback when the schema is too dynamic to compile — for example, when field names depend on runtime configuration.

Two outcomes deserve first-class treatment rather than being lumped into "parse failure":

• Refusal. A schema-constrained response can still be a refusal. Providers may expose this as a distinct field or finish reason. Handle it as a typed outcome, not as a malformed response.
• Truncation. A length-limited response can be a valid prefix of a valid object. Detect it via finish reason, not by trying to parse and failing.

If a malformed response costs you a database write, use constrained decoding. If it costs you a retry, prompt-only JSON with a validator is defensible. If it costs you a customer, add semantic validation regardless of mechanism.

And sometimes the answer is: do not use structured output at all. Free-form prose, exploratory extraction where you do not yet know the shape, and cases where a deterministic parser would beat the model outright are all legitimate reasons to skip the whole apparatus.

Validation as a Two-Stage Gate

Schema validation and semantic validation are separate stages with different failure signatures and different repair strategies. Collapsing them into one boolean is the most common design mistake I see.

Stage one — structural. Parse the text, then validate against the schema. This is cheap, deterministic, and the only stage constrained decoding can make near-certain. It catches syntax errors, missing fields, type drift, and enum violations.

Stage two — semantic. Referential integrity (does this ID exist?), cross-field consistency (does end_date follow start_date?), unit and range sanity, and business invariants the schema cannot express. This stage requires your application's context and cannot be delegated to the model.

The critical operational difference: semantic failures must not be retried the same way as structural failures. A model that produced a valid shape with a wrong value will often reproduce the same wrong value under a naive retry. The shape was fine; the extraction was wrong. Retrying with the same input and the same prompt tends to produce the same guess. Structural failures, by contrast, often resolve on a second attempt with a clearer error message.

A second subtlety: validation library behavior is not absolute. Some validators do not enforce format checks by default — you have to enable them explicitly. Validator coverage of the full JSON Schema specification is imperfect, and it is possible, though rare, for a schema-noncompliant output to pass validation. A passing validation is strong evidence, not proof.

Design your validator to return structured, field-level errors rather than a single boolean. The error object is the input to repair, so its shape determines how good repair can be.

{
  "valid": false,
  "errors": [
    {
      "path": "due_date",
      "constraint": "format",
      "expected": "YYYY-MM-DD",
      "received": "next Friday"
    },
    {
      "path": "priority",
      "constraint": "enum",
      "expected": ["low", "medium", "high"],
      "received": "urgent"
    }
  ]
}

That error object is worth more than any retry count. It tells the repair step exactly what to fix.

Repair Before You Retry

Repair and retry are usually conflated. They are different operations with different costs, and — more importantly — different safety profiles.

Safe normalization handles defects that are local, mechanical, and lossless. Strip markdown fences. Remove a prose preamble. Fix a trailing comma. Trim whitespace. These transformations preserve every value the model produced. They are deterministic code: no model call, no added latency, no cost. A surprising fraction of production failures are normalization-shaped, and teams burn model calls on them because they never wrote the cleanup step.

Policy-controlled mutation is a different animal. Coercing "42" to 42, dropping an unknown field, inserting a default, or normalizing "next Friday" into a date all change the data. Each one can hide an upstream defect, discard information, or make a malformed producer look successful. These transformations are sometimes correct — but only when your application has explicitly decided they are, and only when the decision is logged.

The distinction matters because the article's central claim is that validation leaves evidence. A repair layer that silently coerces types or drops fields erases that evidence before the validator ever sees it. You end up with a pipeline that reports a 99% success rate and a database full of quietly wrong values.

My rule: normalization runs by default; mutation runs only under an explicit, logged policy. If a transformation is lossy or ambiguous, fail or retry instead of repairing. The original raw output is preserved on every attempt, so you can always reconstruct what the model actually said.

Retry handles failures that local fixes cannot reach. The output is structurally or semantically wrong in a way that requires regeneration. Send the original input plus the specific validation error — not the whole conversation again. Replaying the full context invites the model to reproduce the same mistake.

The error message matters more than the retry count. "invalid output" teaches nothing. "field due_date must match YYYY-MM-DD, received 'next Friday'" gives the model a target.

Bound the loop explicitly:

• Max attempts. Two or three is usually enough. If the third attempt fails, the problem is not transient.
• Per-attempt timeout. A slow generation is not a retry candidate; it is a latency problem.
• Hard stop that escalates. Do not loop indefinitely. Escalate to a fallback path.

The cost model is simple: each retry is a full generation. Retry rate is a line item, not a rounding error. A retry rate above a few percent is worth investigating as a schema or prompt problem — but treat that as an alert threshold you calibrate from your own baseline, not a universal constant.

Recovery States and Escalation

A boolean success/failure return forces every caller to reinvent the same branching logic. Name the states explicitly and map each validation outcome to exactly one state and one allowed action.

State	Trigger	Allowed action
accepted	Structural and semantic validation pass	Return to caller
repairable	Lossless normalization fixes the defect	Re-validate; do not count as a retry
retryable	Structural failure that survives normalization	Regenerate with the specific error
needs_context	Semantic failure caused by missing source data	Retrieve or clarify; do not blind-retry
refused	Provider signals a refusal	Surface as a typed outcome; do not parse
escalated	Attempts exhausted or policy forbids repair	Fall back, return typed failure, or route to review

The needs_context state is the one teams miss. A semantic failure often means the model did not have the information it needed — the source text was ambiguous, the referenced record was not in the prompt, or the question required a lookup. Retrying the same prompt with the same context will produce the same guess. The correct action is to fetch the missing data or ask a clarifying question, not to regenerate.

Escalation paths when the loop exhausts:

• Fall back to a weaker but deterministic extractor (regex, a smaller model, a cached result).
• Return a typed failure to the caller so it can decide.
• Route to human review.

Never return a partially valid object. A half-populated record that passes a loose check is worse than a clean failure, because it corrupts downstream state silently.

If the structured output triggers a side effect — a write, a message, a payment — the retry must not double-fire it. Idempotency is part of the contract.

Diagnostics That Survive the Incident

Observability is part of the design, not an afterthought bolted on after the first incident.

Log the raw model output on every attempt, not just failures. The interesting case is the one that passed validation and was still wrong. Without the raw output, you cannot reconstruct what the model actually said.

Record enough to reconstruct the run: attempt number, which stage failed, the field-level error, the model version, the schema version, and a prompt hash. When something breaks at 2 a.m., you want to replay the exact conditions, not guess at them.

Metrics that actually move decisions:

Metric	What it tells you
First-pass validation rate	Baseline reliability of the current schema and prompt
Normalization rate	How much of the failure load is mechanical and lossless
Mutation rate	How often policy-controlled transformations fire
Retry rate	How often regeneration is needed
Escalation rate	How often the pipeline gives up
Field-level error distribution	Which specific field is the problem

A rising retry rate on one field is a schema or prompt bug, not model noise. The field-level distribution tells you which one. If due_date accounts for 80% of errors, the problem is the date format instruction or the source text, not the model's general capability.

Schema versioning matters more than it looks. When the contract changes, old logs become unreadable unless the version is stored alongside the output. A log entry that says "validation failed" is useless if you cannot tell which schema it failed against.

Privacy and retention are design decisions. Raw outputs may contain user data. The logging policy is not a default you inherit — it is a choice you make, and it should be made before the first incident, not during it.

A Minimal Reference Pipeline

Here is the smallest useful version. No framework, no abstraction — just the control flow, with the normalization/mutation boundary made explicit.

def extract(input_text, schema, max_attempts=3):
    prompt = build_prompt(input_text, schema)

    for attempt in range(max_attempts):
        raw = generate(prompt)
        log_attempt(attempt, raw)  # raw preserved on every attempt

        # Safe normalization: lossless, deterministic, always allowed
        cleaned = normalize(raw)

        # Policy-controlled mutation: only if explicitly enabled
        if mutation_policy_enabled(schema):
            cleaned = apply_mutations(cleaned, schema, log=log_mutation)

        # Stage one: structural
        parsed, struct_errors = parse_and_validate(cleaned, schema)
        if struct_errors:
            log_errors(attempt, "structural", struct_errors)
            prompt = build_repair_prompt(input_text, raw, struct_errors)
            continue

        # Stage two: semantic
        sem_errors = validate_semantics(parsed)
        if sem_errors:
            log_errors(attempt, "semantic", sem_errors)
            if is_missing_context(sem_errors):
                return Failure(state="needs_context", errors=sem_errors)
            return Failure(state="escalated", errors=sem_errors)

        return Success(parsed)

    return Failure(state="escalated", attempts=max_attempts)

The loop is short enough to own directly. Framework abstractions — typed model classes, automatic schema generation, built-in error handlers — help when you have many schemas or many providers. They hide the mechanism when you have one. Start with the loop, add abstraction when the repetition earns it.

A worked trace of one failure:

Attempt 1 raw:
  "Here's the extracted data:\n```json\n{\"name\": \"Acme\", \"due_date\": \"next Friday\"}\n```"

Normalize: strip prose preamble, strip fences
  → {"name": "Acme", "due_date": "next Friday"}

Structural validation: pass (both fields present, both strings)
Semantic validation: fail
  → due_date "next Friday" does not match YYYY-MM-DD
  → state: needs_context (relative date requires a reference point)

Attempt 2 prompt includes:
  "field due_date must match YYYY-MM-DD, received 'next Friday'.
   Today's date is 2026-05-08."

Attempt 2 raw:
  {"name": "Acme", "due_date": "2026-05-15"}

Structural: pass. Semantic: pass. Return Success.

Notice that the first attempt passed structural validation. The schema said due_date was a string, and it was. Only semantic validation caught the problem. This is why the two-stage gate matters — a single boolean would have let this through. And notice that the fix was not a blind retry: the second attempt supplied the missing context (the reference date), which is what the needs_context state is for.

A deliberate drill: take one existing extraction prompt, break the schema on purpose — remove a required field, change an enum value, add a type mismatch — and observe which stage catches each break. Then add a lossy mutation to your repair layer and confirm that the original raw output is still in the logs. You will learn more about your pipeline's actual behavior in ten minutes than in a week of reading about it.

Relevant control flow:
if sem_errors:
    if is_missing_context(sem_errors):
        return Failure(state="needs_context", errors=sem_errors)
    return Failure(state="escalated", errors=sem_errors)

The Decision Rule

If the output crosses into typed code, it is untrusted input. The contract is enforced in the validator, not in the prompt. Constrained decoding raises the floor on syntax and conformance; it does nothing for semantics. Normalization handles lossless mechanical failures cheaply; mutation runs only under explicit policy; retry handles the rest, bounded and instrumented. Diagnostics are what turn a 2 a.m. page into a five-minute fix.

Your next action: instrument first-pass validation rate on one existing pipeline, then inspect the field-level error distribution before changing any prompt or schema. The distribution will tell you where the contract is actually failing. Fix that field first.