Make your AI agent state its assumptions

When my coding agent needs a premise it does not have, it does not stop and ask me. It picks the most plausible one and carries on. A value it half-remembers. A convention it assumes the project follows. A claim about what some existing routine does, based on the routine’s name rather than its body. The paragraph it writes on top of that premise reads exactly like one written on a checked premise. Same confidence, same tidy structure. The code compiles either way.

The premise itself is the part that worries me. It never lands anywhere I can look at it. It exists inside one reply, does its work, and is gone by the next message. Six weeks later there is a file, and the file is either right or quietly wrong, and nothing in it tells you which.

So I wrote a rule into the project config file that Claude Code reads at the start of every session. It is short, and it sits under the heading that says these are the non-negotiable ones. An assumption has to be written down, as an assumption, before anything is built on top of it.

The thing that made me write it

There is a piece of numerics at the centre of my simulations. A long-range interaction sum: everything else in the physics leans on it, and if it is wrong then nothing downstream is worth reading.

Early on, it looked right, and looking right was allowed to stand. Nobody wrote down what had actually been verified about it, to what precision, or against which reference, and whatever checking did happen was never recorded anywhere. Over the following months it turned into a black box. Not broken, as far as anyone knew. Just untrusted, in the specific sense that nobody could point at a document and say what had been established.

The cost of that arrives later and it is dull. Trusting a result from it means re-deriving the thing from scratch, which is precisely the work the original shortcut was avoiding. Nothing dramatic happened at any point. No bad paper, no late-night bug hunt. It degraded quietly, which is the only way this failure ever happens.

That is also why the rule lives in a config file and not in my own head as a resolution to be more careful. Care is a real resource and it runs out. Mine runs out around six on a Friday, and Friday evening is exactly when I am tempted to accept something that looks fine.

The rule, in four moves

Paraphrased, because the real one names paths that are nobody’s business:

## Assumptions discipline (hard rule)

Before applying any assumption to code or written output, state it
explicitly and examine it. Non-negotiable for anything that affects
correctness.

- Name it. When a task rests on an assumption, say so plainly before
  acting on it. Never bake it in silently.
- Test it against a primary source, not memory or plausibility: the
  actual code, the project's own notes, the cited literature. Say
  which source confirmed or refuted it.
- If it is load-bearing and cannot be verified, mark it UNVERIFIED
  and ask, rather than proceeding on "looks right".
- Show the check. A benchmark, a cross-read of the derivation, a diff
  against a known-correct reference. State up front the tolerance it
  has to meet.

The one line that carries the whole thing is the third bullet. “Looks right” is not validation.

That sounds like a slogan until you notice how self-tests work. A model that assumed the wrong convention will write a test that encodes the same wrong convention, then run it, then report green. The test inherits the assumption it was supposed to check. I have written about a test of mine that stayed green for months while proving nothing, and that one was my own doing, not an agent’s. A passing check is only evidence when you know what it compares against. If the reference came from the same head as the code, you have measured consistency and called it correctness.

What it looks like in practice

The output shape is what changed. Work is not finished until the assumptions are enumerated somewhere a person can read them, usually at the top of the plan or the note the work produced. It ends up looking like this:

## Assumptions

1. The validation target is the bimodal model's freezing
   temperature, Tc = 1.120(4), not the Gaussian model's 0.951(9).
   Source: Katzgraber, Körner & Young, Phys. Rev. B 73, 224432
   (2006). CONFIRMED, and the two were the wrong way round in
   our own code.
2. `normalise_input` strips trailing whitespace before hashing.
   Source: read the function body, not the docstring. CONFIRMED.
3. The nightly export runs after the backfill completes.
   UNVERIFIED. Nothing in the scheduler config enforces the order.

The first entry is the sort that used to slip past me, so it is worth unpacking. There is a standard toy model of a disordered magnet, the Edwards-Anderson model, and it comes in two flavours that differ only in how the interactions between neighbouring spins are drawn: one takes two values, the other comes from a bell curve. Both freeze into a glassy state at a temperature the literature has pinned down to three decimal places, and the two temperatures are not the same. My code checks itself by reproducing one of them, which only works if it is aiming at the right one.

They were the wrong way round. Each number is individually correct and well known, they were simply attached to the wrong flavour, and nothing about the code looks wrong when you read it. This is the self-test trap from a moment ago, arriving through a number instead of a convention: point the check at the lower value when it should be the higher one, and it will agree with itself all the way down. What catches it is not cleverness. It is one line naming the paper the number came from, so that checking takes ten seconds instead of an afternoon.

The last entry is the other half of the point. It is not verified, it is marked as not verified, and now it is a thing to squint at rather than a thing nobody knew was in the room.

Three times it earned its keep

Start with the dullest one. A symbol went from a meeting into a plan with one letter wrong, and that letter meant a different quantity. It would have travelled into a figure, then into a write-up, then into a conversation with my supervisor, and been caught eventually in the most embarrassing way available. What surfaced it was not clever. The plan had a line flagging it as unverified, my eye landed on that line, and that was the whole mechanism. Half the value of this discipline is that it produces a short list of things to look at twice.

Then there is the documentation that was accurate and still misled me. A set of routines in my package generates sample geometries. They model the shapes and how they pack, and along that path the magnetism is switched off entirely. The documentation described what those routines do, correctly, and never said what they leave out. I had assumed the physics I care about was in there, partly because the everyday name for the procedure makes it sound as though it must be. Describing what a thing does is not the same as bounding what it covers, and a reader fills every unstated gap with whatever they expected to find. The rule that came out of it was to state the exclusions next to the mechanism, in the same paragraph. What is switched off, and what that makes the tool valid for.

The one that annoyed me most was a written note losing an argument to an artefact. An assumption about how an earlier run had been carried out came from a note made at the time by a competent person, and it was simply wrong. The truth was sitting in the run’s own output, where the tooling records the settings automatically, precisely so the question can be settled without anyone’s memory getting involved. The rule that followed: when a note and an artefact disagree, the artefact wins, and the note gets corrected the same day. That goes for my notes and for the agent’s memory files equally.

The rule I could not make executable

I have argued on this blog that when you correct an agent twice, you should stop writing the rule down and turn it into a check that fails the edit. A font size below a floor, a banned word in a label: a hook on every write catches those, and the model fixes itself in the same turn.

This rule is the exception, and I would rather say so than pretend the two are the same shape. No script can tell whether an assumption was verified. There is no pattern match for “did you read the source, or did you recall it”. The text in the file looks identical either way, which is the original problem showing up one level higher.

So it is enforced the only way a rule like this can be, by changing the required shape of the output. The assumptions have to be in the file. Their presence is checkable by a machine, in the crude sense that a section either exists or does not. Their honesty is checkable by me, at a glance, in about a minute. That is worse than a hook and enormously better than hoping.

Why this is not a physics problem

None of this is about simulations. Anyone letting an agent touch a database migration, a pricing calculation or a contract has the same exposure: plausible premises entering the work silently, at speed, leaving no artefact behind. The defence costs a few lines in a config file and some patience with slightly longer replies.

The part I did not expect is how much it helps on the human side. Enumerated assumptions are the fastest review surface I have. Reading five stated premises takes a minute. Reconstructing five unstated ones from the code takes an afternoon, and you will get one of them wrong.