Retries with back-off

Some failures are temporary: the per-minute rate limit (429), an internal error (500), or the model service being briefly unreachable (502, 503, or upstream_error with a 5xx status). Retrying these after a short, growing wait usually succeeds. Everything else (400, 401, 402, 403, 413) will keep failing until you change something, so never resend it as is.

What to retry

Status	Retry?	Wait
`429`	Yes	The `Retry-After` header (seconds), plus a little jitter
`500`, `502`, `503`	Yes	Exponential: about 1 s, 2 s, 4 s, 8 s, capped near 30 s
Other 5xx with `upstream_error`	Yes	Exponential
Network error, timeout, connection reset	Yes	Exponential
`stream_error` event or a stream without `[DONE]`	Yes, the whole request	Exponential
`400`, `401`, `403`, `405`, `413`, 4xx `upstream_error`	No	Fix the request or the key
`402 quota_exhausted`	No	See Handle 402 quota_exhausted

import os
import random
import time
 
import requests
 
URL = "https://app.syntic.ai/v1/chat/completions"
HEADERS = {"Authorization": f"Bearer {os.environ['SYNTIC_API_KEY']}"}
RETRYABLE = {429, 500, 502, 503, 504}
 
 
def complete(body: dict, max_attempts: int = 5) -> dict:
    for attempt in range(max_attempts):
        try:
            r = requests.post(URL, headers=HEADERS, json={**body, "stream": False}, timeout=(10, 300))
        except (requests.ConnectionError, requests.Timeout):
            r = None
        if r is not None and r.ok:
            return r.json()
        if r is not None and r.status_code not in RETRYABLE:
            raise RuntimeError(f"{r.status_code} {r.text}")           # not worth retrying
        if attempt == max_attempts - 1:
            raise RuntimeError(f"gave up after {max_attempts} attempts")
        retry_after = r.headers.get("Retry-After") if r is not None else None
        wait = float(retry_after) if retry_after else min(30, 2 ** attempt)
        time.sleep(wait + random.uniform(0, 0.5))                       # jitter spreads out retries
 
 
reply = complete({"model": "Amara Quick", "messages": [{"role": "user", "content": "Ping"}]})
print(reply["choices"][0]["message"]["content"])

const URL = 'https://app.syntic.ai/v1/chat/completions'
const RETRYABLE = new Set([429, 500, 502, 503, 504])
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms))
 
export async function complete(body: Record<string, unknown>, maxAttempts = 5): Promise<any> {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    let res: Response | null = null
    try {
      res = await fetch(URL, {
        method: 'POST',
        headers: { Authorization: `Bearer ${process.env.SYNTIC_API_KEY}`, 'Content-Type': 'application/json' },
        body: JSON.stringify({ ...body, stream: false }),
        signal: AbortSignal.timeout(300_000),
      })
    } catch {
      res = null // network error or timeout: retryable
    }
    if (res?.ok) return res.json()
    if (res && !RETRYABLE.has(res.status)) throw new Error(`${res.status} ${await res.text()}`)
    if (attempt === maxAttempts - 1) throw new Error(`gave up after ${maxAttempts} attempts`)
    const retryAfter = Number(res?.headers.get('retry-after'))
    const wait = retryAfter > 0 ? retryAfter * 1000 : Math.min(30_000, 1000 * 2 ** attempt)
    await sleep(wait + Math.random() * 500)
  }
}

# curl retries 429 and 5xx itself (honouring Retry-After) with --retry.
curl --retry 5 --retry-max-time 120 --retry-connrefused \
  https://app.syntic.ai/v1/chat/completions \
  -H "Authorization: Bearer $SYNTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "Amara Quick", "stream": false, "messages": [{"role": "user", "content": "Ping"}]}'

Using the OpenAI client libraries

The openai packages for Python and Node.js already retry connection errors, 429 and 5xx responses twice by default with back-off. Raise the count when you create the client:

client = OpenAI(base_url="https://app.syntic.ai/v1", api_key=os.environ["SYNTIC_API_KEY"], max_retries=5)

const client = new OpenAI({ baseURL: 'https://app.syntic.ai/v1', apiKey: process.env.SYNTIC_API_KEY, maxRetries: 5 })

Avoid self-inflicted 429s

The limit is 600 requests per minute per API key. Retries alone cannot fix sustained overload; limit concurrency instead. A simple semaphore around your calls (for example 8 in flight at once) keeps a batch job below the cap. On a 429, read Retry-After for how long until a slot frees up. See Rate-limit headers.

⚠️

Every attempt that reaches the model is billed for the tokens it produced. A failed stream retried three times can cost up to three partial replies. Cap attempts and log each one with its x-syntic-request-id.

Cancel a stream Handle 402 quota_exhausted

Was this page helpful?

Retries with back-off

What to retry

Using the OpenAI client libraries

Avoid self-inflicted 429s

Related