model_busy
The message text is shown in the example below.
| HTTP status | 429 |
type | rate_limit_error |
| Retry? | Yes, with back-off after Retry-After |
| Status | In the public catalog (error-envelope.ts) |
| Extra fields | retry_after |
When it happens
The model service is limiting requests.
Example
{
"error": {
"type": "rate_limit_error",
"code": "model_busy",
"message": "The model is busy right now. Retry shortly.",
"param": null,
"request_id": "req_6f1c2a9e0b7d4c3a8e5f1b2d",
"retry_after": 5
}
}How to fix it
- Retry after
Retry-Afterwith back-off. - If it persists, route non-critical traffic to a different model.
Detect it in code
err = resp.json().get("error", {})
if resp.status_code == 429 and err.get("code") == "model_busy":
delay = err.get("retry_after") or 2
time.sleep(delay) # then retryRelated
Was this page helpful?