Refusals and cut-offs

A schema guarantees the shape of a finished reply. Two situations end a reply before it is a finished object, and your code should handle both.

The model declines

If the request asks for something the model will not do, it does not invent a schema-shaped answer. Instead:

{
  "choices": [{
    "message": { "role": "assistant", "content": null, "refusal": "I can't extract personal data from this document." },
    "finish_reason": "refusal"
  }]
}

Check finish_reason before parsing content. Treat a refusal as a business outcome (route to a person, log it), not as an error to retry.

The reply hits max_tokens

{ "choices": [{ "message": { "content": "{\"items\": [{\"sku\": \"7731-B\", \"qty\": 4}, {\"sku\": \"77" }, "finish_reason": "length" }] }

The content is cut off and will not parse. Fixes, in order of preference:

1. Raise max_tokens. Estimate: about 12 tokens per short field, more for prose fields.
2. Bound arrays with maxItems and process large inputs in pages.
3. Shorten free-text fields with a length rule in their description.

Validation retries

When constrained generation is not available for a schema feature, the server validates the reply and retries once (see JSON output). If the second attempt also fails, the request ends with 422 output_schema_violation, and only the tokens of the attempts are billed. If you see this error regularly, the schema is usually the cause: simplify it.

A safe handler

choice = resp.choices[0]
if choice.finish_reason == "refusal":
    return Outcome.declined(choice.message.refusal)
if choice.finish_reason == "length":
    raise RetryWithMoreTokens()
data = json.loads(choice.message.content)