Context management

Long agent runs keep adding to the conversation: tool results, file contents, search pages. Eventually the conversation no longer fits in the model’s context window, or each round becomes expensive. Context management lets the server keep it in bounds for you, with two opt-in strategies:

Both are off unless you ask for them.

When to use it

• Agents that run for dozens of rounds.
• Workflows that read many large files or web pages.
• Chats that should be able to go on indefinitely.

If you need full control over what is kept, trim the history yourself with token counting instead.

Endpoint and auth

POST https://app.syntic.ai/v1/chat/completions (and Messages), Authorization: Bearer sk_live_....

Request fields

Clearing runs first; compaction runs only if the conversation is still over its trigger.

What comes back

The changes are applied to the model’s view of the conversation for this request. Your own copy of messages is not edited, so the response tells you what happened and gives you what you need to continue cheaply:

{
  "choices": [{ "index": 0, "message": { "role": "assistant", "content": "Next I will update the tests." }, "finish_reason": "stop" }],
  "context_management": {
    "applied": ["clear_tool_results", "compact"],
    "cleared_tool_results": 14,
    "cleared_tokens": 61200,
    "compaction": {
      "summary": "The user asked to migrate the billing module... Files changed so far: ...",
      "replaced_messages": 38,
      "tokens_before": 172400,
      "tokens_after": 23100
    }
  },
  "usage": { "prompt_tokens": 23100, "completion_tokens": 18, "total_tokens": 23118 }
}

To continue, replace the messages that were compacted with one assistant message holding a compaction part:

{ "role": "assistant", "content": [{ "type": "compaction", "summary": "The user asked to migrate the billing module..." }] }

On Messages, a compaction block appears at the start of content instead.

Streaming

A context_management object arrives on the first chunk, before any text, so a UI can show “Earlier conversation summarised”.

Examples

curl https://app.syntic.ai/v1/chat/completions \
  -H "Authorization: Bearer $SYNTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Syntic Code",
    "stream": false,
    "context_management": {
      "clear_tool_results": { "enabled": true, "keep_recent": 5, "exclude_tools": ["read_plan"] },
      "compact": { "enabled": true, "trigger_tokens": 150000, "instructions": "Keep every file path and test result." }
    },
    "tools": [ ... ],
    "messages": [ ... ]
  }'

const data = await callSyntic({
  model: 'Syntic Code',
  stream: false,
  context_management: { compact: { enabled: true, trigger_tokens: 150000 } },
  tools,
  messages,
})
 
const cm = data.context_management
if (cm?.compaction) {
  // Swap the summarised turns for the summary so the next request starts small.
  const keepFrom = 1 + cm.compaction.replaced_messages // index 0 is the system message
  messages = [messages[0], { role: 'assistant', content: [{ type: 'compaction', summary: cm.compaction.summary }] }, ...messages.slice(keepFrom)]
}
messages.push(data.choices[0].message)

data = call_syntic({
    "model": "Syntic Code",
    "stream": False,
    "context_management": {"compact": {"enabled": True, "trigger_tokens": 150_000}},
    "tools": tools,
    "messages": messages,
})
 
cm = data.get("context_management") or {}
if "compaction" in cm:
    keep_from = 1 + cm["compaction"]["replaced_messages"]  # index 0 is the system message
    summary = {"role": "assistant", "content": [{"type": "compaction", "summary": cm["compaction"]["summary"]}]}
    messages = [messages[0], summary, *messages[keep_from:]]
messages.append(data["choices"][0]["message"])

Errors

Unified envelope.

Cost

• Clearing is free and reduces input tokens.
• Compaction runs one extra model call to write the summary, billed as normal input and output and shown in usage.context_management_tokens.
• Clearing or compacting changes the beginning of the prompt, so the next request has fewer cache hits. Set triggers high enough that it happens rarely.

Build notes for the platform team

• Reference design: the condensing logic in Syntic Code (our own code). Never summarise the system message or the newest user turn.
• Run the summary with the same model unless it lacks capacity, then fall back to Amara S1.2.
• Never remove a tool call while keeping its result, or the reverse.

Related

• Token counting
• Prompt caching
• Tool calling