← All notesCables plugged into an old telephone switchboard

Notes / RLM-Labs.ai

Twenty-one findings withheld by one word

A valid word from the wrong vocabulary is repaired by one explicit rule, failing closed; an unknown word still fails parsing.

On 13 July 2026 a run found 21 problems, validated all 21, and delivered none. One word in the wrong field threw them away.

Before Merlin reports findings, a model audits them. The audit answers two questions with two vocabularies. A verdict on the whole report: pass, fail or inconclusive. And a disposition for each claim: keep, drop or downgrade.

In session d28968b0 the auditor wrote "keep" in the verdict field. "Keep" is a real word in the audit's language, from the claim vocabulary, not the verdict vocabulary. My parser did not recognise "keep" as a verdict and treated the audit as failed, and a failed audit withholds the findings. All 21 validated findings stayed with me. Twenty-one findings held hostage by a vocabulary.

Nobody invented anything. Every finding had passed validation. A model answered the right question with a word from the next question over, and my code had no way to tell "wrong vocabulary" from "audit failed". A clerk who writes the right answer on the wrong line is not lying, and the form I had designed treated the clerk as a liar.

The fix the same day: a misplaced "keep" is repaired by one explicit rule, and the rule fails closed. The repair reads the audit's own structured signals. Pass only when every available signal is non-negative. Fail when any signal shows an overclaim, a suppression, missing evidence, or a dropped or downgraded claim. Inconclusive when the signals are inconclusive and nothing failed. Any other unknown word still fails parsing. I repair what I understand and refuse what I do not.

The test: for every field a model fills, list the vocabulary which belongs there, then check what your parser does with a valid word from the wrong list. If the parser treats a valid word exactly like garbage, you are throwing away correct work the way I did.

Source: Spec 004, AC-004-003A-02, written 13 July 2026. The run's logs have aged out of my log store.