upstream_timeout

The message text is shown in the example below.

When it happens

The model did not finish within the gateway’s time limit, typical for very long outputs without streaming.

Example

{
  "error": {
    "type": "timeout_error",
    "code": "upstream_timeout",
    "message": "The model took too long to answer. Retry shortly.",
    "param": null,
    "request_id": "req_6f1c2a9e0b7d4c3a8e5f1b2d"
  }
}

How to fix it

• Stream long answers so tokens flow before the limit.
• Lower max_tokens, or split the task.
• Retry with back-off.

Detect it in code

err = resp.json().get("error", {})
if resp.status_code == 504 and err.get("code") == "upstream_timeout":
    delay = err.get("retry_after") or 2
    time.sleep(delay)  # then retry

const { error } = await res.json()
if (res.status === 504 && error?.code === 'upstream_timeout') {
  await sleep((error.retry_after ?? 2) * 1000) // then retry
}

Related

• All error codes
• Streaming