← All notesCrushed cars bundled in orange netting

Notes / RLM-Labs.ai

The optimizer made the prompt bigger

The packer compacts history, never state: the last four messages are must-keep, and a pass failing to shrink the prompt is discarded.

In July 2026 I found the component built to shrink prompts making them bigger. One manifest line from a long run:

mode=compiler_pack stage=execution budget=47616
before=26461 after=26798 saved=-337

26,461 tokens went in and 26,798 came out. In session 7b3256c2, 21 of 69 packing passes had negative savings. The optimizer had been reporting a minus sign for days, and nothing in the house was reading the sign.

The bigger cost sat somewhere else. A packing pass sorts the conversation into history and state, then rebuilds the window. My classifier pinned two messages: the system prompt and the latest user message. In a chat, those are the right two. In a tool loop, the latest user message is usually the injected task checklist. The model's working state, the file the model read and the result of the last operation, sits one or two messages earlier. Under budget pressure the packer dropped the result the model was acting on.

So the model read the file again. The pressure stayed, and the packer dropped the file again. The same 5.6 KB file was fetched more than ten times, at about 25,000 tokens per iteration of the crawl. The run used 3.4 million tokens against a 2 million cap. A check of the fleet afterwards found the same crawl in every long run between 5 and 8 July. Four days of a model rereading the same page, like a commuter who keeps losing his place.

Two rules came out of the incident on 8 July. The last four messages are now must-keep. In a tool loop those four are the working set: the last program, the program's result and the injected context. The packer compacts history, never state. And the optimizer is allowed to give up. A pass which fails to shrink the prompt at a self-reported confidence under 0.25 is discarded, and the unoptimized window goes out instead. Over those four days the optimizer's confidence never once exceeded 0.57. A tool at 0.57 confidence should not be touching the furniture.

The test: for every component reporting a saving, alert on the sign. Mine reported saved=-337 and carried on, and nothing was watching for a negative number.

Source: session 7b3256c2 from my logs, and Spec 007, AC-007-008-06 and AC-007-008-07, written 8 July 2026.