The test that passed for months was lying

One of the tests in my simulation package had been green since the day I wrote it, months earlier. Every run, every change: pass. Then I changed the order the tests ran in, nothing else, and it failed with two energies hundreds apart. The reorder had not broken the test. It had exposed it. The test had never checked anything at all.

The tidy-up that broke a green test

The package is simulation code I develop for my PhD in computational magnetism. It is a Monte Carlo code: it explores the states of a magnetic system by proposing millions of small random changes and accepting or rejecting each one by a simple rule, and it runs on NVIDIA graphics cards (CUDA). Its test suite grew the way research test suites usually grow: around twenty standalone files, each written next to the feature it checks, each run by hand whenever that feature felt wobbly. No shared harness.

During a repository tidy-up (the same kind of unglamorous housekeeping as centralising your plot styling, and similarly more consequential than it looks) I gave the suite an aggregated runner. One process, every test file in sequence, one summary at the end. No test logic changed. Logistics only, I thought.

First full run: one failure. The failing test was an equivalence test. It drives two implementations of the same simulation update, a straightforward one and an optimised one, and requires them to produce exactly the same run: the same sequence of simulated states (the trajectory), identical to the last bit. Under the aggregated runner they disagreed at once. Not a rounding question. Different physics.

Then the confusing part. Run the same file on its own and it passed, cleanly, every time. Whole suite: red. Standalone: green. Reproducible in both directions all afternoon.

What standalone-pass, aggregate-fail is telling you

That combination is a specific symptom, and it is worth learning to read. When a test’s verdict depends on what ran before it, something is reading state it does not own. Global variables, the internal state of random number generators, caches, files on disc, whatever earlier code left behind in memory. Changing the order changed that shared state, and the shared state changed the verdict. The only question is which shared state.

In my case: GPU memory, in a way I would not have guessed.

The mechanics of the lie

The equivalence test builds two fresh simulation states, runs one implementation on each, and compares trajectories. What I had missed, months ago, was one call. The test never ran the initialisation routine that fills in the state’s temperature lookup. So the update kernel (the small program that actually runs on the graphics card) looked up a temperature index that had never been set, and used it to read one element past the end of the array holding the inverse temperatures (1/T, the form the formulas use).

Why does a temperature sit at the heart of this at all? Because temperature is the knob the whole simulation exists to study. A Monte Carlo sampler accepts or rejects each proposed spin flip with a probability set by the energy change and the inverse temperature: cold runs are picky and mostly keep moves that lower the energy, hot runs accept nearly anything. Run the same system at different temperatures and the outcomes should differ; that dependence is the physics being measured. Which is exactly what makes a corrupted temperature read so quiet. A kernel fed garbage in place of the inverse temperature does not crash or complain. It runs a perfectly plausible simulation of some arbitrary temperature nobody asked for.

On an ordinary processor with bounds checking on (the automatic check that an index really lies inside its array), that is an immediate error with a message pointing at the offending line. On a graphics card it is nothing of the sort. An out-of-bounds read does not crash. The device fetches whatever bytes happen to live at that address and hands them over as if they were data. You get a number. Often the same number every run, so the garbage looks deterministic and respectable.

Now both behaviours explain themselves. Standalone, in a fresh process, the two simulation states were placed in untouched graphics-card memory, so both out-of-bounds reads landed on the same leftover value. Both implementations simulated at the same nonsense temperature, produced the same nonsense trajectory, and “are the two trajectories bit-identical?” sailed through. The test was comparing garbage against an exact copy of the same garbage. It had been doing that since the day it was written.

Under the aggregated runner, other tests had already used and released that memory. The two states now landed on different leftovers, read different garbage, and the trajectories split. That failure was the first honest thing this test ever did.

The fix was one initialisation call per state. The comparison now tests what it was always meant to test.

Two identical failures also match

The general lesson is about comparison tests, and it stings. A comparison test, reference against optimised, old against new, proves exactly one thing: both sides did the same thing. “The same thing” includes “the same wrong thing”. If neither side exercises the property you care about, the comparison passes vacuously, and it will keep passing for as long as both sides fail identically.

So a comparison test needs each side to prove it is alive before agreement means anything. That proof can be cheap. Assert that the output responds to the inputs it is supposed to respond to: my trajectories should change when the temperature changes, and a trajectory driven by an uninitialised lookup would not have. Or pin one side to a small case you can check by hand. Either would have caught this on day one.

The second bug, same afternoon

Once everything ran under one harness, I moved the suite to the standard entry point of Julia’s package manager, Pkg.test. Among other things, Pkg.test forces bounds checking back on, overriding the @inbounds markers that switch it off in production runs for speed. It is slower. It found a second bug within minutes.

The routine that builds a histogram, sorting each measured distance into its bin, was writing one slot past the end of its array. Not in an exotic corner case: in every ordinary run. The shape of the bug is generic enough to write out, because I suspect versions of it live in a lot of analysis code. Something like this:

# nbins bins of width dr, covering distances up to r_max
if r < r_max                        # guard on the radius
    bin = floor(Int, r / dr) + 1
    hist[bin] += 1                  # can still hit hist[nbins + 1]
end

The radius guard and the index guard are not the same condition. r < r_max is a statement about a float. bin <= nbins is a statement about an integer produced by dividing two floats and flooring the result. With perfect numbers the two conditions coincide. With the slightly rounded numbers computers actually store, whenever r_max / dr is not quite what you think it is, a thin sliver of radii passes the first test and violates the second. Guard the thing you actually index with:

bin = floor(Int, r / dr) + 1
if 1 <= bin <= nbins                # guard on the index
    hist[bin] += 1
end

The model that wrote it missed it; the model that reviewed it did not

Full disclosure: this package is AI-assisted research code. The original implementations, including the equivalence test that lied, were written months ago in sessions with Claude Opus, and every run since kept them green. Both bugs surfaced in a single repository consolidation pass with Claude Fable 5, a newer and stronger model, running at a high reasoning-effort setting.

I do not think that was raw cleverness alone. Writing a feature and auditing a codebase are different postures, and the consolidation pass had licence to restructure: the aggregated runner and the stricter bounds-checked entry point it built for boring housekeeping reasons are what mechanically exposed the bugs. But it did take judgement to look at a months-green test failing after a mere reorder and conclude that the standalone PASS was the artefact, not the new runner. The reviewing model made that call straight away and traced it to the missing initialisation.

The takeaway if you use AI on research code: whichever model writes your code shares your blind spots at writing time. Budget a separate adversarial pass, with the strongest model you can point at the problem, whose only job is to break what the writing sessions built. Mine paid for itself within the afternoon, twice.

Would a human have done better?

A fair question, because it is tempting to read the previous section as “AI writes buggy tests”. So suppose there were no AI anywhere in this story, just a person writing that same test by hand, without a deep feel for how graphics cards handle memory. Does the same bug slip through silently?

Almost certainly, and the record says so. To avoid it, the author had to hold three facts in mind at once. That building a simulation state is a two-step affair, and the test skipped step two. That the update kernel reaches through the missing second step to look up its temperature. And that a graphics card, asked to read past the end of an array, does not stop or warn, it hands back whatever bytes happen to be lying there. People have been tripping over each of these for decades. Whole categories of tools exist precisely because human-written code makes these mistakes all the time: Valgrind on ordinary processors, Compute Sanitizer on NVIDIA cards. Nobody builds an industry of memory checkers around a mistake that nobody makes.

Where a person may actually fare worse is not in making the mistake but in reading the failure. “Passes alone, fails in the suite” has a well-worn human response: blame the new test runner, call the test flaky, add a retry, or pin the tests back to the old order so everything goes green again. Freezing the order is common enough in automated testing setups to count as a named bad habit. Every one of those responses buries the bug for another few months.

What AI changes is volume, not kind. A model produces convincing-looking tests much faster than a person does, so unexamined ones can pile up faster too. But the defence is the same whoever wrote the code, and it is mechanical rather than heroic: shuffle the order, switch the safety checks on while testing, run a memory checker now and then, and make every comparison test prove both sides are alive before trusting their agreement.

Cheap habits that catch this class of bug

Green is not tested. A month of passes tells you the assertions held in one particular execution context. It says nothing about whether they could ever fail, and an assertion that cannot fail is decoration.

Reordering tests is close to a free fuzzer. Execution order is an input to your suite, and running in one fixed order forever means sampling a single point of it. An aggregated runner that changes execution context is a feature here, not a hazard. Any test whose verdict flips with order has found you a defect, in the code or in the test itself, and either one is worth knowing about today rather than after the results go out.

Run the entry point that re-arms the safety checks your production configuration disables, even when it is slower. Fast and unchecked for producing data, slow and checked for testing. That split is the whole point of having two modes.

And on a GPU, treat “it did not crash” as zero information. The device will read past your arrays all day and never say a word. When a kernel result surprises you, suspect memory before physics.

If your research code has a pile of standalone test files, here is a half-hour experiment: run them all in one process, then run them again in reverse order. I expected that exercise to produce a tidier summary printout. It produced two real bugs, one of which had spent months hiding behind a green tick. The tests that change their answer are the interesting ones.