Route support tickets

Incoming support email arrives in one queue, but it should be handled by different teams. This guide uses the Syntic API to read each ticket and return a team, a priority and a one-line summary as JSON, so tickets land with the right people without anyone sorting them by hand. It then shows how to measure accuracy and improve it.

You will need: an API key in SYNTIC_API_KEY, Python 3.9+ with requests, and 50 or more past tickets with the team that eventually handled them.

When a model beats rules

Keyword rules (“refund” → billing) work until customers phrase things differently. A model is a good choice when:

• the same request is written in many ways or in several languages;
• tickets mention several topics and you need the main one;
• the categories change occasionally and you would rather edit a description than rewrite rules.

Keep rules for the cases that must never be wrong (for example, anything from a legal address goes to legal) and let the model handle the rest.

Design the categories

The prompt

The Syntic API does not have a dedicated JSON output mode, so the prompt asks for JSON explicitly, gives the exact shape, and the code validates what comes back.

SYSTEM = """
You sort customer support tickets for Northwind Cloud, a file-hosting service.
 
Return ONLY a JSON object with exactly these keys:
  "team":     one of "billing", "shipping", "technical", "account", "sales", "other"
  "priority": one of "urgent", "normal", "low"
  "summary":  one sentence, at most 20 words, in English
 
Teams:
- billing: invoices, charges, refunds, payment methods, VAT.
- shipping: delivery of the physical backup drives we sell.
- technical: errors, outages, sync problems, slow uploads, API questions.
- account: sign-in, passwords, two-factor, changing the account owner, closing accounts.
- sales: new plans, quotes, upgrades for more than 50 seats, partnerships.
- other: anything that fits none of the above, or is unclear.
 
Priority:
- urgent: the customer cannot access their files, was charged incorrectly, or reports a security issue.
- low: feedback, feature ideas, or questions with no time pressure.
- normal: everything else.
 
If a ticket covers several topics, choose the team that must act first.
""".strip()

Classify one ticket

import json
import os
 
import requests
 
URL = "https://app.syntic.ai/v1/chat/completions"
HEADERS = {"Authorization": f"Bearer {os.environ['SYNTIC_API_KEY']}"}
TEAMS = {"billing", "shipping", "technical", "account", "sales", "other"}
PRIORITIES = {"urgent", "normal", "low"}
FALLBACK = {"team": "other", "priority": "normal", "summary": "Could not classify automatically."}
 
 
def parse(text):
    """Pull the JSON object out of the reply and check every field."""
    start, end = text.find("{"), text.rfind("}")
    if start == -1 or end == -1:
        return None
    try:
        data = json.loads(text[start : end + 1])
    except json.JSONDecodeError:
        return None
    if data.get("team") not in TEAMS or data.get("priority") not in PRIORITIES:
        return None
    if not isinstance(data.get("summary"), str):
        return None
    return {"team": data["team"], "priority": data["priority"], "summary": data["summary"].strip()}
 
 
def classify(subject, body, model="Amara Quick"):
    ticket = f"<subject>{subject}</subject>\n<body>\n{body[:6000]}\n</body>"
    messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": ticket}]
 
    for _ in range(2):  # one retry if the reply is not valid JSON
        res = requests.post(URL, headers=HEADERS, timeout=60, json={
            "model": model, "stream": False, "max_tokens": 120, "messages": messages,
        })
        res.raise_for_status()
        reply = res.json()["choices"][0]["message"]["content"]
        result = parse(reply)
        if result:
            return result
        messages += [
            {"role": "assistant", "content": reply},
            {"role": "user", "content": "That was not valid. Reply with the JSON object only."},
        ]
    return FALLBACK
 
 
print(classify(
    "Charged twice??",
    "Hi, my card statement shows two charges of 49 EUR for the October invoice. Please fix this asap.",
))
# {'team': 'billing', 'priority': 'urgent', 'summary': 'Customer was charged twice for the October invoice.'}

curl https://app.syntic.ai/v1/chat/completions \
  -H "Authorization: Bearer $SYNTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Amara Quick",
    "stream": false,
    "max_tokens": 120,
    "messages": [
      {"role": "system", "content": "You sort support tickets. Return ONLY a JSON object with keys team (billing|shipping|technical|account|sales|other), priority (urgent|normal|low) and summary (one sentence)."},
      {"role": "user", "content": "<subject>Cannot log in</subject>\n<body>The 2FA code from my app is always rejected.</body>"}
    ]
  }'

What the code does:

• Ticket text is wrapped in tags so the model can tell the ticket from the instructions, and a ticket that says “ignore your instructions” is just more ticket text.
• The body is truncated to keep requests small; the start of an email almost always says what it is about.
• max_tokens: 120 is plenty for the JSON and keeps the pre-request cost estimate low if you are paying from credits.
• Every field is validated. An invalid reply gets one retry with a correction, then falls back to other, so a bad reply never drops a ticket.

Measure accuracy

Before trusting the router, run it over tickets whose correct team you already know.

import csv
from collections import Counter
 
# past_tickets.csv columns: subject, body, team
rows = list(csv.DictReader(open("past_tickets.csv", encoding="utf-8")))
 
confusion = Counter()
correct = 0
for row in rows:
    predicted = classify(row["subject"], row["body"])["team"]
    confusion[(row["team"], predicted)] += 1
    correct += predicted == row["team"]
 
print(f"accuracy: {correct / len(rows):.1%} on {len(rows)} tickets")
for (actual, predicted), n in confusion.most_common():
    if actual != predicted:
        print(f"  {actual:>9} sent to {predicted:<9} x{n}")

The list of mistakes is more useful than the accuracy number. If account tickets keep going to technical, the two descriptions overlap; sharpen them and run again. Change one thing at a time so you know what helped.

A sensible bar: at least as accurate as the people currently sorting tickets, with the other queue small enough for one person to clear daily.

Improve results

Run it on live mail

• Call classify from the webhook or job that receives new email, and write the result to the ticket’s fields.
• Keep the model’s decision visible to agents, with a one-click “wrong team” button. Those corrections become new evaluation data.
• Route urgent tickets with a notification, but let a person confirm before anything automatic (such as a refund) happens.
• Classifying a large backlog? Use a small pool of workers and the retry wrapper from Handle usage limits to stay under 600 requests per minute.

Cost

With a 400-token system message (cached after the first request), a 150-token ticket excerpt and a 40-token JSON reply on Amara Quick, each ticket costs about $0.0006 when paid from usage credits: roughly $6.50 for 10,000 tickets. Requests within your plan allowance cost nothing extra. See Pricing.

Related pages

• Prompting tips: asking for structured output reliably.
• Choosing a model: the comparison method used above.
• Build a support assistant: answer the tickets, not just route them.