Chain prompts

One request that expects the model to research, analyse, decide and write all at once tends to do each part a little worse than a prompt that does one of them. Chaining splits the work into a sequence of requests, where each request has one job and passes its output to the next. Your code sits between the steps and can check, filter or branch on each result.

When to chain

Chain when you see any of these:

The prompt has several distinct sub-tasks with different instructions.
One part of the output is consistently weaker than the rest.
You need to check something in code between steps (a total, a policy, a lookup).
Different steps suit different models: a cheap one for sorting, a stronger one for writing.

Keep a single prompt when the task is simple; every extra step adds latency and a little cost.

Example: from customer interview to product brief

A product team turns raw interview transcripts into a brief. As one prompt, the result mixes quotes with interpretation and misses themes. As a chain:

transcript ─▶ [1 extract]  quotes tagged by topic          (Amara Quick)
           ─▶ [2 cluster]  themes with supporting quotes   (Amara S1.2)
           ─▶ [3 check]    code: every quote exists in the transcript
           ─▶ [4 write]    one-page brief from the themes   (Amara S1.2)

import json
import os
 
from openai import OpenAI
 
client = OpenAI(base_url="https://app.syntic.ai/v1", api_key=os.environ["SYNTIC_API_KEY"])
 
 
def ask(model: str, system: str, user: str, max_tokens: int = 1500) -> str:
    r = client.chat.completions.create(
        model=model, stream=False, max_tokens=max_tokens,
        messages=[{"role": "system", "content": system}, {"role": "user", "content": user}],
    )
    if r.choices[0].finish_reason == "length":
        raise RuntimeError(f"step output cut off ({model})")
    return r.choices[0].message.content or ""
 
 
def as_json(text: str):
    return json.loads(text[text.find("["): text.rfind("]") + 1])
 
 
def interview_brief(transcript: str) -> str:
    # 1. Extract: pull out verbatim quotes. Cheap model, narrow task.
    quotes = as_json(ask("Amara Quick",
        'Extract every statement where the interviewee describes a problem, a workaround or a wish. '
        'Reply with a JSON array of {"quote": verbatim text, "topic": 2-4 words}.',
        f"<transcript>\n{transcript}\n</transcript>"))
 
    # 3 (run early). Check: drop quotes that are not verbatim, before anything builds on them.
    flat = " ".join(transcript.split()).lower()
    quotes = [q for q in quotes if " ".join(q["quote"].split()).lower() in flat]
 
    # 2. Cluster: group quotes into themes.
    themes = as_json(ask("Amara S1.2",
        'Group these interview quotes into 3-6 themes. Reply with a JSON array of '
        '{"theme": short name, "summary": one sentence, "quotes": [quote texts]}. Use only the quotes given.',
        json.dumps(quotes)))
 
    # 4. Write: the brief, from themes only.
    return ask("Amara S1.2",
        "Write a one-page product brief in Markdown for a product team: a two-sentence overview, "
        "then one section per theme with the summary and up to two supporting quotes. No recommendations.",
        json.dumps(themes))

Patterns

Pattern	Shape	Example
Pipeline	A ─▶ B ─▶ C	Extract, then cluster, then write (above).
Map, then reduce	many A in parallel ─▶ B	Summarise each chapter, then summarise the summaries.
Router	A decides ─▶ B1 or B2	Classify a ticket, then use the prompt for that category.
Draft and review	A writes ─▶ B critiques ─▶ A revises	Marketing copy checked against brand rules.
Verify	A answers ─▶ B checks against sources	See Reduce hallucinations.

Pass structured data between steps

Hand-offs between steps are where chains break. Prefer JSON or tagged text that the next step and your code can both read, and validate it between steps. A step that receives malformed input should fail loudly rather than improvise.

Run independent steps in parallel

In a map step, the requests do not depend on each other. Run them concurrently (for example with asyncio.gather or Promise.all), keeping concurrency below the per-key cap (600 calls a minute). See Retries with back-off for handling the occasional 429.

Debugging a chain

Log the input and output of each step with its request id (x-syntic-request-id). When the final output is wrong, walk backwards until you find the first step whose output is wrong; that is the prompt to fix. This is much faster than debugging one large prompt, because each step’s output can be judged on its own.

Each step is billed separately by its usage. Steps that resend the same large document benefit from caching if the document sits at the very start of each request.

Build an agent: when the model, not your code, decides the next step.
Summarize long documents: map and reduce in practice.
Test your prompts

Keep a persona Prompt for tool use

Was this page helpful?