Streaming

A streamed reply arrives in many small pieces as the model writes it, so your interface can show text right away rather than after the whole answer is ready. Streaming is the Syntic API’s default: any request that does not send "stream": false is streamed.

When to stream

To turn it off, send "stream": false. Leaving the field out means streaming, which differs from many other chat APIs.

The wire format

The response uses server-sent events (Content-Type: text/event-stream). Each event is a line that starts with data: followed by a JSON chunk, and events are separated by a blank line. The stream ends with a literal data: [DONE].

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Pack"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" light"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":21,"completion_tokens":40,"total_tokens":61}}

data: [DONE]

Syntic always asks the model service to report usage on streamed requests, so you do not need to request it. The usage totals arrive on a chunk near the end, which may have an empty choices array. Code that reads choices[0] without checking will fail on that chunk.

Read a stream with an SDK

Any OpenAI-compatible SDK can read the stream once you point it at the Syntic base URL.

import os
from openai import OpenAI
 
client = OpenAI(base_url="https://app.syntic.ai/v1", api_key=os.environ["SYNTIC_API_KEY"])
 
stream = client.chat.completions.create(
    model="Amara S1.2",
    stream=True,
    messages=[{"role": "user", "content": "Give me a packing list for a two-day hike."}],
)
 
parts = []
usage = None
for chunk in stream:
    if chunk.usage:
        usage = chunk.usage
    if chunk.choices and chunk.choices[0].delta.content:
        piece = chunk.choices[0].delta.content
        parts.append(piece)
        print(piece, end="", flush=True)
 
print()
print("tokens used:", usage.total_tokens if usage else "unknown")
full_reply = "".join(parts)

import OpenAI from 'openai'
 
const client = new OpenAI({ baseURL: 'https://app.syntic.ai/v1', apiKey: process.env.SYNTIC_API_KEY })
 
const stream = await client.chat.completions.create({
  model: 'Amara S1.2',
  stream: true,
  messages: [{ role: 'user', content: 'Give me a packing list for a two-day hike.' }],
})
 
let fullReply = ''
let usage
for await (const chunk of stream) {
  if (chunk.usage) usage = chunk.usage
  const piece = chunk.choices[0]?.delta?.content ?? ''
  fullReply += piece
  process.stdout.write(piece)
}
process.stdout.write('\n')
console.log('tokens used:', usage?.total_tokens ?? 'unknown')

# -N turns off curl's output buffering so chunks print as they arrive.
curl -N https://app.syntic.ai/v1/chat/completions \
  -H "Authorization: Bearer $SYNTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"Amara S1.2","messages":[{"role":"user","content":"Give me a packing list for a two-day hike."}]}'

Read a stream without an SDK

If you would rather not add a dependency, read the response body yourself. Split it on line breaks, keep any unfinished line for the next read, and parse each data: line:

const res = await fetch('https://app.syntic.ai/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SYNTIC_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'Amara S1.2',
    messages: [{ role: 'user', content: 'Explain tides to a ten-year-old.' }],
  }),
})
 
if (!res.ok) {
  // Errors before the stream starts are ordinary JSON.
  const err = await res.json()
  throw new Error(`${res.status} ${err.error}`)
}
 
const reader = res.body.getReader()
const decoder = new TextDecoder()
let pending = ''
 
outer: while (true) {
  const { value, done } = await reader.read()
  if (done) break
  pending += decoder.decode(value, { stream: true })
  const lines = pending.split('\n')
  pending = lines.pop() // last item may be an incomplete line
 
  for (const line of lines) {
    if (!line.startsWith('data:')) continue
    const payload = line.slice(5).trim()
    if (payload === '[DONE]') break outer
    const chunk = JSON.parse(payload)
    if (chunk.error) throw new Error(`stream failed: ${chunk.error.code}`)
    process.stdout.write(chunk.choices?.[0]?.delta?.content ?? '')
  }
}

Tool calls in a stream

When the model calls a function during a streamed reply, the call arrives in pieces under delta.tool_calls. Each piece carries an index that says which call it belongs to. The first piece for a call has its id and function.name; later pieces add fragments of function.arguments, which only form valid JSON once joined.

calls = {}  # index -> {"id", "name", "arguments"}
 
for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta
    for part in delta.tool_calls or []:
        slot = calls.setdefault(part.index, {"id": None, "name": "", "arguments": ""})
        if part.id:
            slot["id"] = part.id
        if part.function and part.function.name:
            slot["name"] += part.function.name
        if part.function and part.function.arguments:
            slot["arguments"] += part.function.arguments
    if chunk.choices[0].finish_reason == "tool_calls":
        break
 
# Now each calls[i]["arguments"] is a complete JSON string.

Parse the arguments only after the stream reports finish_reason: "tool_calls". Then run the functions and continue as described in Tool calling.

Errors during a stream

Problems found before the model starts, such as a bad key, an exhausted quota or the rate limit, come back as an ordinary JSON error with a non-200 status. Check the status code before you start reading events. See Errors.

If something fails after the stream has begun, the HTTP status is already 200. The API sends one more event and then closes the stream:

data: {"error":{"type":"api_error","code":"stream_error","message":"The response stream was interrupted. Retry the request.","param":null,"request_id":"req_6f1c2a9e0b7d4c3a8e5f1b2d"}}

Check each parsed chunk for an error field and treat the reply as incomplete when you see one. A stream that ends without data: [DONE] is also incomplete. In both cases you can retry the request; tokens produced before the failure are still billed.

Stopping early

To stop a reply partway, close the connection: break out of the loop and close the stream (or abort the fetch with an AbortController). The model stops writing soon after.

Response headers

The x-syntic-request-id, x-syntic-billing-source and, when it applies, x-syntic-model-downgraded headers are sent before the first event, exactly as on a non-streamed reply. Read them from the response object before you start iterating.

Planned changes

stream_options.include_usage

Usage will always arrive on its own final chunk, after the chunk with finish_reason, so code can rely on one shape across models. Billing is unaffected: Syntic records usage whether or not you ask to receive it.

Keep-alive comments

During long pauses (a large prompt being read, or a built-in tool such as web search running) the server sends an SSE comment line about every 15 seconds:

: keep-alive

Lines that start with : carry no data. SDKs skip them; hand-written parsers should too (the parser above already does, because it only reads lines that start with data:). Proxies and load balancers see traffic and do not close the connection.

Mid-stream errors in the standard envelope

The final error event will use the same body as HTTP errors (see Error format) and will never contain raw text from the model service:

data: {"error":{"type":"overloaded","code":"stream_interrupted","message":"The reply was interrupted. Retry the request.","param":null},"request_id":"req_01JBXC9"}

During a transition period the old stream_error event continues to be sent to keys pinned to an earlier API version. Check for both shapes: typeof chunk.error === 'string' (old) or chunk.error.code (new).

Example with the new options

curl -N https://app.syntic.ai/v1/chat/completions \
  -H "Authorization: Bearer $SYNTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"Amara S1.2","stream":true,"stream_options":{"include_usage":true},
       "messages":[{"role":"user","content":"Give me a packing list for a two-day hike."}]}'

Build notes for the platform team

• Replace {error:'stream_error', detail} in api-proxy with the envelope; map exceptions to the codes above and log the raw text server-side only.
• Emit : keep-alive from a timer that resets on every write.
• Always split usage into its own chunk, regardless of what the model service sends.

Related

• Stream format
• Stop reasons
• Messages streaming events