Batches
⚠️
Preview — this feature is being built; the design below is the spec.
A batch runs up to 50,000 chat or Messages requests asynchronously within 24 hours at 50% of the usual token rates. Submit the requests, wait (or receive a batch.completed webhook), then download one results line per request.
| Operation | Method and path |
|---|---|
| Create a batch | POST /v1/batches |
| List batches | GET /v1/batches |
| Get a batch | GET /v1/batches/{batch_id} |
| Cancel a batch | POST /v1/batches/{batch_id}/cancel |
| Download results | GET /v1/files/{output_file_id}/content |
Input line
{"custom_id": "ticket-1001", "endpoint": "/v1/chat/completions", "body": {"model": "Amara Quick", "max_tokens": 20, "messages": [{"role": "user", "content": "Label this ticket: My parcel never arrived."}]}}Result line
{"custom_id": "ticket-1001", "result": {"type": "succeeded", "status_code": 200, "body": {"choices": [{"index": 0, "message": {"role": "assistant", "content": "delivery"}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 21, "completion_tokens": 2, "total_tokens": 23}}}}result.type is succeeded, errored, canceled or expired; only succeeded lines are billed.
Status lifecycle
status | Meaning |
|---|---|
validating | The input is being checked. Invalid lines fail the whole batch here. |
in_progress | Requests are running. |
canceling | Cancel requested; running requests finish. |
completed | Every line finished. |
expired | The 24-hour window ended first; unfinished lines are marked expired. |
canceled | Cancelled; finished lines are in the results. |
failed | Validation failed; nothing ran or was billed. |
The full walkthrough, with input preparation and polling code, is in the Batches guide.
Was this page helpful?