Compaction is cheaper than the context it removes

When a coding agent’s context fills up, it gets compacted: the history is replaced by a summary and the session continues. Everyone who has watched this happen mid-task has the same instinct — compact as late as possible. The summary loses detail, and compacting throws away the prompt cache you have been paying to build. Better to hold on.

Half of that is true. The other half is expensive, and I had the threshold set wrong for weeks because of it.

The cache does not survive the boundary

Cached input tokens cost a tenth of uncached ones, so a long stable prefix looks cheap. It is cheap per token. You just pay for it on every single turn.

I went through twelve compaction boundaries in my transcripts and compared the cache figures on either side. Before a boundary, cache reads ranged from 93k to 950k tokens per turn. After one: 15k to 31k. The next few turns then show large cache writes — 34k to 403k — as the cache is built again from the summary.

Only the stable prefix survives a boundary. Everything the session actually accumulated is gone from the cache, because it is gone from the context. This is by design, and it means “delay compaction to preserve the cache” is not a trade-off at all. Delaying compaction does not save the cache. It only keeps you paying full price for a large context on every turn until you compact anyway.

So compact sooner

Claude Code lets you set the context size at which it compacts automatically — autoCompactWindow in ~/.claude/settings.json, or the /autocompact command for one session. Mine was at 500k. I moved it to 250k.

Comparing the week before the change with the two days since — sessions split by whether they started before or after the threshold moved, main sessions across every project, 43k assistant turns on one side and 6.6k on the other:

Narrowing the baseline to the morning of the change — same day, same kind of work, a much smaller sample — gives the same picture: 1.00 to 0.73 per turn, median 237k to 150k, p90 389k to 244k.

The number that convinced me is not in that table. I broke the week’s spend down by the context size it was spent at, and the top band — turns running above 500k tokens of context — had been taking between 14% and 31% of the money, every single day. On the first full day after the change it was zero. That band is pure overhead: the same work, done while dragging a much larger history behind it.

What it cost

More compactions, obviously. Over the first thirty-two hours: 23 boundaries in my main sessions, 20 of them automatic, and 6 more inside subagents. The automatic ones fired between 217k and 250k tokens rather than exactly 250k, since the threshold leaves room for the turn that follows.

That is about double what I had modelled. Replaying the previous week’s context traces against a 250k threshold predicted one boundary per 170 turns; the real rate is nearer fifteen a day, almost all of it in a single heavy project.

Each compaction is one model call over the whole context: roughly $0.5, and a median of 163 seconds. Twenty-nine of them cost about $15 and an hour of waiting across those thirty-two hours. The money is not the interesting half — at a 250k window a compaction pays for itself within about five turns of the smaller context that follows, and the sessions where this matters run hundreds of turns; at 950k it pays for itself in under two. The wall clock is the half worth knowing before you copy the setting: it comes to something like forty minutes a day of waiting, not the couple of minutes I expected.

The lesson generalises past the specific number. Compaction is a fixed cost paid once. A large context is a variable cost paid every turn. Whenever you find yourself protecting the second to avoid the first, check the arithmetic.

What breaks when you compact more often

Summaries keep decisions and lose reasons.

“We chose A” survives compaction. “We rejected B because we measured it and it was worse” does not — it reads as history, and history is what a summary compresses first. So the next session proposes B again, and you pay for that experiment twice. Compacting three times as often means meeting that failure three times as often.

Claude Code has a PreCompact hook that runs before the summary is written, and whatever the hook prints on stdout is added to the instructions the model gets for writing it. Mine prints this:

When writing the compaction summary, add a dedicated section titled
"Rejected alternatives and why" and fill it from the conversation being
compacted. For every approach, design, library, schema, tool or debugging
route that was considered and then dropped, record:

  - what the option was;
  - why it was rejected — keep the concrete reason: the number that was
    measured, the error that reproduced, the constraint that blocked it.
    Never flatten a reason to "did not work" or "was not suitable";
  - what was chosen instead;
  - under what condition the decision would be worth revisiting.

Two things about this hook are worth knowing before you write one, both of which cost me a compaction to find out. This is Claude Code 2.1.236; hook plumbing changes between versions.

Print plain text, not JSON. The usual way for a hook to inject context is a JSON object with hookSpecificOutput.additionalContext. PreCompact has no such field — the schema defines it for other events only, and a JSON payload is rejected at compaction time with a validation error. Raw stdout is the interface here.

Preserve the user’s own instructions. The hook’s output becomes newCustomInstructions, which replaces the text you type in /compact <instructions> rather than appending to it. If your hook prints only its own text, anything you asked for by hand is silently dropped. Read custom_instructions from the hook’s stdin, print it first, then your own:

#!/usr/bin/env bash
set -euo pipefail
dir="$(dirname "$0")"
user_instr="$(jq -r '.custom_instructions // empty' 2>/dev/null || true)"
if [ -n "$user_instr" ]; then
  printf '%s\n\n' "$user_instr"
fi
cat "$dir/precompact-preserve.txt"

It does layer onto the built-in summary prompt rather than replacing it — the summaries I get still have every standard section, plus the injected one. Note that this is the opposite of the API’s server-side compaction, where the similarly-named instructions field replaces the default prompt entirely. Two close names, two different behaviours.

The cost is about a thousand tokens per boundary, and the section chains: items carried into one summary survive into the next, as long as they are still on the input.

What this does not fix

A cheaper session is not a better one. Compacting sooner means the model sees less of its own history sooner, and there are tasks where that hurts — a long review that has to hold many findings at once, for instance. For those I raise the window back up for that one session and pay for it deliberately, which is the difference between an expensive session and an expensive week.

And the hook only rescues reasoning that was written down in the session. It does not survive the session ending. That is a different problem, and I wrote floppy for it.