Thirty thousand characters, every turn

The assistant harness I use most sends a standing instruction block of roughly thirty thousand characters with every turn. Another one I have been testing sends about nine thousand. A third sends almost nothing.

My first instinct was that the big ones were bloated. Reading them properly, that is wrong. The content is good: how to break work into steps, when to search before editing, how to avoid destroying things. Advice I would give.

The problem is that all of it arrives before the model knows what it has been asked. It is paid on the turn where I want a one line answer and on the turn where I am restructuring a subsystem, identically. The instruction about how to explore an unfamiliar codebase is present while I am asking what time a backup runs.

So I have been working on the other shape. Keep a minimal base, and serve the rest at the moment of need. It sounds obvious written down, and it is not the way any of these systems are built, mine included.

The first piece I moved was memory. Before the model sees my question, my own notes are searched and the matching one is placed into the conversation. Not a standing instruction telling the model to check its memory. The note, in the conversation, before the answer.

The result was better than I expected because of what I tested it on. I ran it against a small, cheap, fast model, the kind that ignores instructions when it is confident. I asked where my daily backups are kept. It answered with the correct path, cited the identifier of the note it came from, and made no tool calls at all. It did not check its memory. It did not need to. The memory was already in front of it.

Then I went to read how the harness actually delivers this, and found the property that makes the whole approach work.

The hook I attach to fires before each model call and hands me the list of messages. What it hands me is a copy. Whatever I add is used for that one call and then discarded; the real conversation, the thing that gets saved and later compacted, never contains it. I had assumed I was writing into the session and would have to be sparing.

That changes the economics completely. A system prompt is paid on every turn for the rest of the session, and worse, it stays in the record and gets carried through compaction, so a bad line is expensive twice. This is paid on the turn that uses it and then gone. When the subject changes, it does not linger, it simply is not there.

It also has one design consequence I got wrong first. My initial version attached the note to the final message in the list. That works for the opening call of a turn and then the note vanishes, because the moment the model runs a tool the final message is the tool’s output, not my question. Present for the first call and absent for the ten that do the actual work. The fix is to attach it to my question wherever my question sits, which means it survives the whole turn and disappears when I ask the next one.

The uncomfortable part is what this implies about the rest.

If I want the minimal harness, I do not get to simply delete the thirty thousand characters. That block is doing work. Take it away and the model stops breaking tasks into steps, stops exploring before editing, stops the small disciplines that make it useful. Removing it is not subtraction, it is a transfer: whatever it was providing becomes mine to provide, at the right moment, in a form the model can act on.

That is a much larger commitment than it looks, and it is why I am doing it one piece at a time. Take out one thing. Replace it with something served just in time. Measure whether the behaviour survived. Only then take out the next. The failure mode I want to avoid is the one where I strip the scaffolding, feel clever about the token count, and quietly get a worse assistant that nobody measured.

There is a version of this that goes further, and I keep circling it. The interesting thing was never that memory arrives on time. It is that a system can decide, per turn, what its own operator needs to have in front of it, and be wrong in ways that are measurable. Every piece I move out of the standing block becomes a decision the system makes rather than a paragraph it recites. That is more fragile and much more alive, and I would rather debug something alive.