Stream format

Unless a request sets "stream": false, Chat completions returns its reply as server-sent events (SSE). This page describes exactly what arrives on the wire and how to turn it back into a message. For when and why to stream, see Streaming.

The response

Status 200 OK is sent before generation starts. Errors found before that point (bad key, quota, invalid body) arrive as ordinary JSON error responses with a non-200 status, never as a stream.
Headers: Content-Type: text/event-stream, Cache-Control: no-cache, no-transform, plus x-syntic-request-id, x-syntic-billing-source and, when relevant, x-syntic-model-downgraded.
Body: a sequence of lines. A line that starts with data: carries one event. A blank line ends each event. Ignore any other line (for example, comment lines that start with :).

Event types

`data:` payload	Meaning
A JSON object with `"object": "chat.completion.chunk"`	The next piece of the reply.
A chunk whose `usage` field is set	Token counts for the whole request. It arrives at or near the end.
`{"error": "stream_error", "detail": "..."}`	Generation failed after the stream began. No more content follows.
`[DONE]`	The stream is complete. This is plain text, not JSON.

Chunk fields

Field	Description
`id`	Same value on every chunk of one reply.
`object`	Always `chat.completion.chunk`.
`created`	Unix time in seconds.
`model`	The model identifier reported for this reply.
`choices[0].delta.role`	`assistant`, usually only on the first chunk.
`choices[0].delta.content`	The next fragment of text. Append fragments in order. May be absent or empty.
`choices[0].delta.tool_calls`	Fragments of tool calls (see below).
`choices[0].finish_reason`	`null` until the last content chunk, then `stop`, `length` or `tool_calls`.
`usage`	`prompt_tokens`, `completion_tokens` and `total_tokens`, on the usage chunk only. Its `choices` array may be empty.

Assembling tool calls

When the model calls a function during a streamed reply, delta.tool_calls arrives spread over several chunks. Each fragment has an index that says which call it belongs to:

The first fragment for a call carries its id, type and function.name.
Later fragments carry more characters of function.arguments.

Collect fragments by index, concatenate the arguments strings, and parse the result as JSON only after finish_reason is tool_calls.

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"tc_01","type":"function","function":{"name":"check_stock","arguments":""}}]},"finish_reason":null}], ...}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"sku\":\"LAV"}}]},"finish_reason":null}], ...}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"-2L\",\"warehouse\":\"leeds\"}"}}]},"finish_reason":null}], ...}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}], ...}

Reading a stream

import json
import os
import requests
 
def stream_reply(messages, model="Amara S1.2"):
    resp = requests.post(
        "https://app.syntic.ai/v1/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['SYNTIC_API_KEY']}"},
        json={"model": model, "messages": messages},  # stream defaults to true
        stream=True,
        timeout=300,
    )
    if resp.status_code != 200:
        body = resp.json()
        raise RuntimeError(f"{resp.status_code} {body.get('error')}")
 
    text, calls, usage = [], {}, None
    for raw in resp.iter_lines(decode_unicode=True):
        if not raw or not raw.startswith("data:"):
            continue
        payload = raw[5:].strip()
        if payload == "[DONE]":
            break
        event = json.loads(payload)
        if event.get("error") == "stream_error":
            raise RuntimeError(f"stream failed: {event.get('detail')}")
        if event.get("usage"):
            usage = event["usage"]
        for choice in event.get("choices", []):
            delta = choice.get("delta", {})
            if delta.get("content"):
                text.append(delta["content"])
                print(delta["content"], end="", flush=True)
            for frag in delta.get("tool_calls", []):
                slot = calls.setdefault(frag["index"], {"id": None, "name": "", "arguments": ""})
                slot["id"] = frag.get("id") or slot["id"]
                fn = frag.get("function", {})
                slot["name"] += fn.get("name") or ""
                slot["arguments"] += fn.get("arguments") or ""
    return "".join(text), list(calls.values()), usage
 
reply, tool_calls, usage = stream_reply(
    [{"role": "user", "content": "Write a two-line poem about tram stops."}]
)
print("\n", usage)

Network chunks do not line up with events. Always buffer until you have a full line, as both examples do, before parsing JSON.

Disconnects and billing

If your client closes the connection before [DONE], generation is not refunded: tokens already produced are billed. When the stream ends before the usage chunk was sent, the bill is estimated from the length of the text delivered and the size of the request. To stop a long reply cheaply, set a sensible max_tokens instead of cutting the connection.

Errors during a stream

A stream_error event means the model service failed part-way through. The text you already received is incomplete. Retry the whole request; do not try to continue from the partial text. Log x-syntic-request-id from the response headers so support can trace the failure.

Delete a template Error codes

Was this page helpful?