Why agents get worse as context grows

An agent that answered well on Monday answers vaguely on Friday. It stops following an instruction it followed twenty messages ago. It calls a tool, gets the right data back, and then writes an answer that ignores half of it. Nothing errors, nothing is logged, and the conversation looks fine. This is usually called context rot, and the common explanation — "the context window filled up" — is not what we measured.

One turn, end to end

A turn starts as one request, grows while the tools run, and is rebuilt before the answer is written. The middle state is the one worth looking at: it is what most agents answer from.

Every turn is one self-contained request

The model keeps nothing between turns. There is no session on the other side, no memory of what it said an hour ago, no way for it to look something up that was not handed to it. Each turn, the platform assembles a complete request and sends it whole; the reply is generated from that request and nothing else.

If you have a networking instinct for this, it is close to a datagram: self-contained, sent once, carrying everything the receiver needs, and bounded by a size the sender must respect. Two things break the analogy, and both matter:

• Nothing is lost in transit. Everything you send arrives. What varies is how much of it the model actually attends to — the loss happens inside the receiver, not on the wire.
• The layout changes the answer. A router does not behave differently because you rearranged the payload. A model does. The same facts, in the same request, arranged two different ways, produced a correct answer in one arrangement and a wrong answer in the other. That result is the reason for most of what follows.

What is in the request

Everything above the history is a single system message. That is deliberate: the agent's boundaries, your knowledge base and the reply language are not conversation, and a model treats them differently depending on where they sit.

What gets dropped when it does not fit

A request that exceeds the budget is not truncated from the end. Blocks are dropped in priority order until it fits:

1. Oldest history first — and the dropped turns are compacted into the running summary rather than discarded, so what was said survives even when the wording does not.
2. Lowest-scoring memories, then lowest-scoring retrieved passages — from the tail, so the most relevant material is the last to go.
3. Profile, then summary.
4. Never the agent's instructions, your knowledge base, or the question. If the question alone overflows the budget it is truncated and marked as truncated — the model is told, rather than left to guess why a sentence stops mid-word.

This ordering is the honest part of a context budget: something has to go, and a system that will not say what will quietly drop whatever happened to be last.

Fitting is not the same as being read

Here is the part that surprised us. We built a test where the facts needed to answer were present in every condition — nothing was dropped, nothing was truncated, no information was missing anywhere. The only thing that changed was how the request was arranged. A filtering task over a few hundred records, run against our own local model and a cloud model, several trials each:

Two controls rule out the obvious readings. Splitting the clean request into two messages of the same total length still scored 8 / 8, so length is not the variable. Handing the distilled rows back as a tool result still scored 0 / 8, so it is not simply that the data needs to sit closer to the question. Tool scaffolding and unrelated material compete for the model's attention wherever they are placed.

The cloud model was more robust but not immune: it held up on most tasks and still dropped to 0 / 3 on a single-value lookup once a turn's accumulated tool results were in the request.

Two honest limits on this. It is our own test on our own tasks, not a public benchmark. And it does not raise a model's ceiling: a harder variant of the same task scored zero under every arrangement, because that particular model could not do it at all. Rearranging a request recovers accuracy the arrangement was costing you; it does not buy capability the model does not have.

Working and answering are two different requests

This is the split the animation at the top of the page is showing.

Working needs the messy request. The model has to see its own tool calls and their raw results to decide whether to call another one. That phase is unchanged.

Answering does not. Once the tools have finished, the platform builds a fresh request — the same system instructions, a short recap of the conversation, the facts this turn actually found, and the question — and the reply you receive is generated from that. The tool calls, the raw payloads and the intermediate reasoning do not appear in it.

The recap is there because a real question is often "yes" or "and the second one?". The single-message arrangement that measured best is a single-turn one, and a conversation is not; the recap is folded into the same message rather than restored as separate turns, which costs nothing measurable and keeps pronouns resolvable.

What we deliberately do not do

The common fix for tool output flooding a context window is to have a model summarise each result before it goes into the conversation. We do not, and the test above is why: the arrangement that scored full marks used the raw data, re-rendered — no summarising step was needed to recover the accuracy. Adding one would mean asking a model to decide which facts matter, in a system whose whole design principle is to not hand a model a job that code can do exactly. A summariser that drops the one number the customer asked about fails silently and looks like a good answer.

When the facts genuinely do not fit, they are truncated and the truncation is stated in the request, with an instruction not to fill in the missing part. An agent that says it could only see the first forty orders is worth more than one that confidently invents the rest.

What this means for you

There is nothing to configure. The behaviour above is how every agent on the platform runs.

What it changes in practice:

• Long conversations stay usable. The twentieth turn is assembled the same way as the first, and what fell out of the window survives as a summary rather than vanishing.
• A tool that returns a lot of data does not poison the reply. The result is capped, and the answer is written from a request that no longer carries the raw payload.
• A turn that used tools costs one extra model call — roughly a second on a short reply. The chat shows what it is doing while that runs, rather than sitting silent.

If you do see an agent ignoring something you know it was given, the useful question is not "is the context window too small" but "which block was it in, and what else was in the request with it". Where a conversation lives between turns answers the other half of this page — if the model keeps nothing, what is on our side that a request gets assembled from. Grounded answers covers what gets retrieved, Per-customer memory what gets remembered, and Tools and MCP the budgets that keep a single tool result from taking the whole window.