Context windows

The context window is the most text a model can handle in one request, counted in tokens: everything you send (system message, conversation, tool definitions, tool results) plus everything it writes back. Every model in the Syntic catalog has a context window of 262,144 tokens.

What counts

 ┌──────────────────────────── 262,144 tokens ────────────────────────────┐
 │ identity line │ system message │ tools │ conversation + tool results │ reply │
 └──────────────── prompt (usage.prompt_tokens) ──────────────────────────┴───────┘
                                                        reply ≤ max_tokens

• Prompt: reported afterwards as usage.prompt_tokens. It includes the short Syntic identity line, your system message, all messages and the tool definitions.
• Reply: reported as usage.completion_tokens and bounded by max_tokens.
• The two together must fit in the window. A prompt of 250,000 tokens leaves at most about 12,000 for the reply.

Budget before you send

Without a counting endpoint, estimate and then correct with real figures:

WINDOW = 262_144
SAFETY = 0.9          # keep 10% headroom: estimates are rough
 
 
def estimate_tokens(messages, tools=None) -> int:
    """Rough estimate: about 4 characters per token for English prose."""
    chars = sum(len(str(m.get("content") or "")) + 20 for m in messages)
    chars += len(str(tools or ""))
    return chars // 4
 
 
def max_reply_tokens(messages, tools=None, wanted=4000) -> int:
    room = int(WINDOW * SAFETY) - estimate_tokens(messages, tools)
    if room < 500:
        raise ValueError("conversation too long: trim or summarise before sending")
    return min(wanted, room)

const WINDOW = 262_144
const SAFETY = 0.9 // keep 10% headroom: estimates are rough
 
export function estimateTokens(messages, tools = []) {
  let chars = messages.reduce((n, m) => n + String(m.content ?? '').length + 20, 0)
  chars += JSON.stringify(tools).length
  return Math.floor(chars / 4) // about 4 characters per token for English prose
}
 
export function maxReplyTokens(messages, tools = [], wanted = 4000) {
  const room = Math.floor(WINDOW * SAFETY) - estimateTokens(messages, tools)
  if (room < 500) throw new Error('conversation too long: trim or summarise before sending')
  return Math.min(wanted, room)
}

The four-characters rule is a rough guide for English prose. Code, tables, JSON and many non-English languages use more tokens per character. After each response, compare your estimate with usage.prompt_tokens and adjust your ratio for your own content.

What happens when it does not fit

A request whose prompt plus max_tokens exceeds the window is refused by the model service. Today it comes back as an upstream_error with a 4xx status and a short description; with the error envelope it becomes invalid_request with a specific code. Separately, requests are capped at about 2 MB of JSON and 500 messages (413 errors), which large conversations can hit before the token limit. See Limits.

If the prompt fits but the reply reaches the end of the window, the reply stops with finish_reason: "length".

Using a large window well

A large window makes it possible to send whole documents, but quality and cost still favour sending what matters:

• Cost: every token in the prompt is billed on every request (at the cached rate when repeated). A 200,000-token prompt sent 50 times is 10 million input tokens.
• Latency: long prompts take longer before the first token appears.
• Focus: a relevant 5,000-token excerpt often gives sharper answers than 200,000 tokens of mostly irrelevant text.

Strategies, roughly from simplest:

1. Send the whole document when it is the subject of many questions, and keep it at the start of the prompt so repeat requests are cached. See Work with long documents.
2. Retrieve only the relevant passages. See Answer from your documents.
3. Summarise in stages (map and reduce). See Summarize long documents.
4. In long conversations and tool loops, trim old turns and tool results. See Tool results and context.

Related pages

• Limits
• Context management
• Pricing