Work with long documents

Every Syntic model has a context window of 262,144 tokens for the prompt and the reply together. That is room for several hundred pages of text in one request. Fitting a document in is the easy part; getting precise answers out of it depends on how you lay it out.

Rough sizes

Token counts vary with language and formatting; tables, code and non-English text use more tokens per word. Check real figures in usage.prompt_tokens on a test request. A dedicated counting endpoint is planned; see Token counting.

The request body is also capped at about 2 MB of JSON, so very large inputs can hit the byte limit before the token limit. See Limits.

Put the documents first and the question last

Place long material at the top of the user message and the instructions and question after it. With long inputs, a question at the end gets better answers than the same question placed before 100 pages of text.

<documents>
  <document index="1">
    <source>supplier_agreement_2026.txt</source>
    <content>
    ...
    </content>
  </document>
  <document index="2">
    <source>amendment_03.txt</source>
    <content>
    ...
    </content>
  </document>
</documents>

Using only the documents above: what notice period applies if we end the agreement early,
and does the amendment change it? Quote the clauses you rely on.

Label each document

Give every document a number and a source (file name, title, URL). The model can then say where an answer came from, and you can check it. Without labels, answers drift into “the document says”, which is impossible to verify across ten documents.

Ask for quotes before conclusions

For questions about long material, ask the model to first copy out the passages that matter, then answer from those passages. Pulling the relevant text forward helps the model ignore the rest and makes errors easy to spot.

import os
from pathlib import Path
 
from openai import OpenAI
 
client = OpenAI(base_url="https://app.syntic.ai/v1", api_key=os.environ["SYNTIC_API_KEY"])
 
docs = [Path("supplier_agreement_2026.txt"), Path("amendment_03.txt")]
blocks = "\n".join(
    f'<document index="{i}">\n<source>{p.name}</source>\n<content>\n{p.read_text()}\n</content>\n</document>'
    for i, p in enumerate(docs, start=1)
)
question = "What notice period applies if we end the agreement early, and does the amendment change it?"
 
r = client.chat.completions.create(
    model="Amara S1.2",
    stream=False,
    max_tokens=1200,
    messages=[
        {"role": "system", "content": "You answer questions about contracts for a procurement team. Use only the documents provided."},
        {"role": "user", "content": (
            f"<documents>\n{blocks}\n</documents>\n\n"
            "First, inside <quotes> tags, copy word for word every clause relevant to the question, "
            "each with its document index and clause number. If none are relevant, write 'none'.\n"
            "Then answer inside <answer> tags using only those quotes.\n\n"
            f"Question: {question}"
        )},
    ],
)
print(r.choices[0].message.content)
print("prompt tokens:", r.usage.prompt_tokens)

# Build the JSON with jq so file contents are escaped correctly.
jq -n --rawfile a supplier_agreement_2026.txt --rawfile b amendment_03.txt '{
  model: "Amara S1.2", stream: false, max_tokens: 1200,
  messages: [
    {role: "system", content: "You answer questions about contracts for a procurement team. Use only the documents provided."},
    {role: "user", content: ("<documents>\n<document index=\"1\"><source>supplier_agreement_2026.txt</source><content>\n" + $a +
      "\n</content></document>\n<document index=\"2\"><source>amendment_03.txt</source><content>\n" + $b +
      "\n</content></document>\n</documents>\n\nQuote the relevant clauses in <quotes>, then answer in <answer>.\nQuestion: What notice period applies if we end early?")}
  ]}' | curl -s https://app.syntic.ai/v1/chat/completions \
    -H "Authorization: Bearer $SYNTIC_API_KEY" -H "Content-Type: application/json" -d @- \
  | jq -r '.choices[0].message.content'

Many questions about the same document

When users ask several questions about one document, keep the document in the same position at the start of every request (system message or first user message) and change only what comes after it. The identical beginning is served from the prompt cache, so follow-up questions cost a fraction of the first. Nothing that varies between requests (the current time, a user id) should come before the document.

Clean the text before sending

• Remove repeated page headers, footers and page numbers that PDF extraction leaves behind; they interrupt sentences and waste tokens.
• Keep headings and numbering. “Clause 14.2” is how people refer to contracts.
• Convert tables to Markdown or CSV rather than flattened text, so rows and columns stay aligned.
• For code, include file paths and keep indentation.

When the material is too large

If the material does not fit, or every question needs only a small part of it:

• Retrieve: search for the relevant passages and send only those. See Answer from your documents.
• Map and reduce: summarise or extract from each section separately, then combine the results in a final request. See Summarize long documents and Chain prompts.

Related pages

• Reduce hallucinations
• Summarize long documents
• Control costs