Stream tool calls
When a streamed reply asks for tools, the calls arrive in fragments inside choices[0].delta.tool_calls, spread over many chunks. Your code joins the fragments by their index and only parses the arguments once the stream has ended with finish_reason: "tool_calls". This page shows the assembly in full, plus how to show progress to the user while it happens.
What the fragments look like
The first fragment for a call carries its index, id, type and function.name. Later fragments carry only the index and a piece of function.arguments:
data: {"choices":[{"index":0,"delta":{"role":"assistant","content":null,"tool_calls":[{"index":0,"id":"tc_01","type":"function","function":{"name":"get_order","arguments":""}}]},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"order_"}}]},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"number\":\"FL-20391\"}"}}]},"finish_reason":null}]}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":512,"completion_tokens":24,"total_tokens":536}}
data: [DONE]When the model asks for several tools at once, fragments for different calls carry different index values and can interleave.
Assemble the calls
import json
import os
from openai import OpenAI
client = OpenAI(base_url="https://app.syntic.ai/v1", api_key=os.environ["SYNTIC_API_KEY"])
def stream_turn(messages, tools, on_text=lambda s: None, on_tool_start=lambda name: None):
"""Streams one turn. Returns (assistant_message, finish_reason)."""
text, calls, finish = [], {}, None
stream = client.chat.completions.create(model="Amara S1.2", stream=True, tools=tools, messages=messages)
for chunk in stream:
if not chunk.choices:
continue
choice = chunk.choices[0]
delta = choice.delta
if delta.content:
text.append(delta.content)
on_text(delta.content)
for frag in delta.tool_calls or []:
slot = calls.setdefault(frag.index, {"id": None, "type": "function", "function": {"name": "", "arguments": ""}})
if frag.id:
slot["id"] = frag.id
if frag.function and frag.function.name:
slot["function"]["name"] += frag.function.name
on_tool_start(slot["function"]["name"])
if frag.function and frag.function.arguments:
slot["function"]["arguments"] += frag.function.arguments
finish = choice.finish_reason or finish
message = {"role": "assistant", "content": "".join(text) or None}
if calls:
message["tool_calls"] = [calls[i] for i in sorted(calls)]
return message, finish
# In a loop: append `message`, then if finish == "tool_calls", parse each
# json.loads(call["function"]["arguments"]) inside try/except, run it, and append tool messages.Show progress while tools run
Users notice silence. Use the moment a tool name arrives to show what is happening:
| Event | Show |
|---|---|
Tool name arrives (on_tool_start) | “Looking up your order…” (map each tool name to a friendly phrase) |
| Your code starts running the tool | A spinner next to that phrase |
| Next turn starts streaming text | Replace the status line with the answer |
Never show raw argument fragments to users; they are incomplete JSON until the stream ends.
Rules that prevent bugs
- Parse arguments only after the stream ends. A partial string such as
{"order_is not valid JSON. - Key by
index, not by position in the chunk. Fragments of different calls can arrive in any order. - Keep the assistant message exactly as assembled, with
content(possiblynull) and the fulltool_calls, when you send it back. - Handle text and tools in one turn. Some turns contain a short text (“Let me check that.”) followed by tool calls. Show the text; still run the tools.
- Check for a stream error. If a
stream_errorevent arrives or[DONE]never does, the tool calls may be incomplete; retry the turn rather than running them.
For tool rounds where nobody watches the text, stream: false is simpler: the reply arrives with complete tool_calls. Many applications stream only the final answer, as in Build a support assistant.