Prompt caching

When a request begins with exactly the same content as a recent request, Syntic reuses the work already done on that shared beginning. Those input tokens are billed at the cheaper cached input rate and the reply starts sooner. Chats and tool loops benefit most, because every round re-sends the conversation so far.

How it works

1. Syntic looks at the start of your prompt in order: system message, tool definitions, then messages.
2. If that beginning matches a prefix seen recently from the same organisation, the matched part is a cache hit.
3. Everything after the first difference is processed normally and becomes available for the next request.

Nothing needs to be marked. Caching never changes the reply, only the cost and speed.

Request field

Endpoint: POST https://app.syntic.ai/v1/chat/completions (or Messages), Authorization: Bearer sk_live_....

Response fields

The cached count has one fixed place:

{
  "usage": {
    "prompt_tokens": 10000,
    "completion_tokens": 800,
    "total_tokens": 10800,
    "prompt_tokens_details": { "cached_tokens": 6000 }
  }
}

Some responses today also carry a top-level usage.cached_tokens. It stays for a transition period with the same value, then is removed under a new API version.

Cost

With Amara S1.2 at $2.50 input and $0.40 cached input per million tokens (see Pricing), a 10,000-token prompt with 6,000 cached costs 4,000 × $2.50 + 6,000 × $0.40 per million = $0.0124 instead of $0.025.

Design prompts for hits

• Put what never changes first: system message, then tools in a fixed order, then long reference material.
• Put what changes every time last: the new question, the date, ids.
• Append to the conversation; editing an earlier message breaks the match from that point.
• Send a prompt_cache_key when many users share the same long instructions.

Examples

curl -s https://app.syntic.ai/v1/chat/completions \
  -H "Authorization: Bearer $SYNTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Amara S1.2",
    "stream": false,
    "prompt_cache_key": "support-bot-v3",
    "messages": [
      { "role": "system", "content": "<4,000-token support handbook here>" },
      { "role": "user", "content": "How do I reset my router?" }
    ]
  }' | jq .usage.prompt_tokens_details

async function ask(question) {
  const resp = await fetch('https://app.syntic.ai/v1/chat/completions', {
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.SYNTIC_API_KEY}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({
      model: 'Amara S1.2',
      stream: false,
      prompt_cache_key: 'support-bot-v3',
      messages: [
        { role: 'system', content: HANDBOOK }, // identical every time
        { role: 'user', content: question },   // changes every time
      ],
    }),
  })
  const data = await resp.json()
  const { prompt_tokens, prompt_tokens_details } = data.usage
  console.log(`cache hit ${prompt_tokens_details.cached_tokens}/${prompt_tokens}`)
  return data.choices[0].message.content
}

import os, requests
 
def ask(question: str) -> str:
    data = requests.post(
        "https://app.syntic.ai/v1/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['SYNTIC_API_KEY']}"},
        json={
            "model": "Amara S1.2",
            "stream": False,
            "prompt_cache_key": "support-bot-v3",
            "messages": [
                {"role": "system", "content": HANDBOOK},  # identical every time
                {"role": "user", "content": question},    # changes every time
            ],
        },
        timeout=60,
    ).json()
    u = data["usage"]
    print(f"cache hit {u['prompt_tokens_details']['cached_tokens']}/{u['prompt_tokens']}")
    return data["choices"][0]["message"]["content"]

In the Console

The Usage page adds a cache hit rate per model and per key (cached input tokens divided by all input tokens), so you can see whether a prompt change helped.

Errors

Unified envelope. Caching itself never fails a request; a cache miss is simply billed at the normal rate.

Build notes for the platform team

• The gateway already derives a cache key from the session id; when the caller sends prompt_cache_key, use org_id: + that value instead.
• Normalise the cached count with readCachedTokens() and always emit it under prompt_tokens_details.
• Add cached_input_tokens to the usage row (already summed for billing) and show the hit rate in the Console.

Related

• Pricing
• Token counting
• Context management