Get long outputs

Reports, translations of whole documents and large code files need more than a few hundred words. Long outputs fail in predictable ways: they stop early, they trail off in quality, or they hit max_tokens.

Set max_tokens deliberately

The reply cannot be longer than max_tokens. Most models allow 32,768 output tokens and Amara S2 65,536. If finish_reason is length, the reply was cut off: raise the limit or split the work.

Plan, then write

Ask for an outline first, check it, then ask for each section in turn, passing the outline each time. Sections stay consistent, and a failure costs one section rather than the whole document.

Say how long

“About 1,500 words, in five sections of roughly equal length” is followed far better than “a detailed report”. Words and sections are easier for models to judge than tokens.

Stream it

Long outputs take time. Stream so users see progress and so you avoid gateway time-outs on non-streamed requests.

Continue after a cut-off

If a reply ends with finish_reason: "length", send it back as the assistant turn and add a user message: “Continue exactly where you stopped.” Check the join for repeated or missing sentences.

Translation and rewriting

For a long document, work in chunks of a few pages, carrying a short glossary of names and terms between chunks so wording stays consistent.