Stream structured output

Structured output streams like any other reply: the JSON text arrives in fragments. Users see progress, and your code can act on fields as soon as they are complete.

What arrives

With stream: true and a json_schema response format, each chunk’s delta.content holds the next piece of JSON text. The concatenation is always valid JSON matching the schema once the stream ends with finish_reason: "stop". Individual chunks are not valid JSON on their own.

Parsing as you go

Use a tolerant parser that completes an unfinished JSON prefix:

import { parse as parsePartial } from 'partial-json'
 
let text = ''
for await (const chunk of stream) {
  text += chunk.choices[0]?.delta?.content ?? ''
  const partial = parsePartial(text) // e.g. { "title": "Refund pol", "steps": [] }
  render(partial)
}

The SDK helpers do this for you: client.chat.completions.stream({..., response_format}) emits content.delta events with a parsed snapshot of the object so far.

Design for streaming

• Put the fields people should see first at the top of the schema; fields are written in schema order.
• For long lists, show items as each one closes rather than re-rendering the whole list on every chunk.
• Do not act on a field (send an email, book a slot) until the stream has finished and the final object validated. A cut-off stream can leave a field half-written.

When the stream breaks

If the stream ends with an error event or finish_reason: "length", the JSON is incomplete. Discard it and retry with a higher max_tokens, or narrow the schema. See Refusals and cut-offs.