The eval grader object

⚠️

Preview — this feature is being built; the design below is the spec.

The eval_grader object as the API returns it. Every field below appears in every response that returns this object; fields with no value are null rather than missing.

Example

{
  "id": "evgr_01JD46E8G0J2L4N6Q8S0",
  "type": "eval_grader",
  "name": "support-rubric",
  "kind": "model_rubric",
  "config": {
    "model": "Amara Deep",
    "rubric": "Score 1-5. 5 = correct facts, apologises once, gives a concrete timeline, under 120 words. Deduct 2 for any invented policy.",
    "pass_threshold": 4
  },
  "created_at": "2026-11-01T09:30:00Z",
  "updated_at": "2026-11-01T09:30:00Z"
}

Fields

Field	Type	Description
`id`	string	Unique id of this object.
`type`	string	The object type, always `eval_grader`.
`name`	string	Name shown in the Console and in reports.
`kind`	string	`exact_match`, `contains`, `regex`, `json_schema`, `similarity` (embedding cosine against `ideal`), or `model_rubric` (a model scores the output against your rubric).
`config`	object	Settings for the kind. For `model_rubric`: `model`, `rubric` and `pass_threshold` on a 1 to 5 scale. For `similarity`: `threshold` from 0 to 1. For `regex`: `pattern`.
`created_at`	string	When it was created, RFC 3339 UTC (some older objects use Unix seconds).
`updated_at`	string	When it last changed.

Returned by

eval_dataset eval_item

Was this page helpful?