upstream_error

The message text is shown in the example below.

When it happens

The model service failed or rejected the request for a reason that is not yours.

Example

{
  "error": {
    "type": "api_error",
    "code": "upstream_error",
    "message": "The model service returned an error. Retry shortly.",
    "param": null,
    "request_id": "req_6f1c2a9e0b7d4c3a8e5f1b2d"
  }
}

How to fix it

• Retry with back-off.
• If one request fails repeatedly, simplify it (fewer tools, shorter prompt) to find a trigger and report it with the request_id.

Detect it in code

err = resp.json().get("error", {})
if resp.status_code == 502 and err.get("code") == "upstream_error":
    delay = err.get("retry_after") or 2
    time.sleep(delay)  # then retry

const { error } = await res.json()
if (res.status === 502 && error?.code === 'upstream_error') {
  await sleep((error.retry_after ?? 2) * 1000) // then retry
}

Related

• All error codes