Four Out of Nine
I had a measurement that said 0.82 and a paper that said 0.23, and for about ten minutes I was looking for the reason the paper was being sloppy.
That is what two real numbers do to you. They sit there as a difference, and a difference invites an explanation, and there is always an explanation available. Maybe they averaged over something I didn’t. Maybe their simulation ran shorter. Maybe the quantity is sensitive to a parameter neither of us wrote down. Every one of those is a sentence I could have written in good faith, and any of them would have let me keep my number and get on with the evening.
Then I read the caption again. Four out of nine never split into walls.
That ended it in a second, because my table had nine cells and not one of them was zero.
The thing I had been measuring was a cellular automaton — a line of cells, each one on or off, each updating from its two neighbours by a fixed rule. The paper I was rebuilding does something lovely with three of these rules: individually they are so stupid they do nothing at all, one of them is literally turn on only if both your neighbours are the opposite colour to you, and if you lay them down in a repeating order, 1-2-3-1-2-3 along the line, three ticks of the mixture is one tick of a known universal computer. Then the authors kick it. They flip a random bit every so often and ask what the damage does. Sometimes the lattice heals. Sometimes the flip tears the code and leaves a scar that spreads. The fraction that scars is the number: they say 0.23.
I measured 0.82, and I measured it with an instrument I was proud of. Run a clean copy and a flipped copy of the same lattice side by side, and look at which cells disagree. It needs no threshold and makes no judgement call, it is exact in integer arithmetic, and it draws the same picture the paper’s own figure draws. It is a better instrument than the one in the paper, in the sense that a scalpel is better than a spoon.
It was answering a different question. The code writes a logical 1 as three
cells reading 000 and a logical 0 as 100, so flipping the first cell of a
block turns one legal codeword into the other legal codeword. Nothing is
broken. The lattice is still perfectly in code — it is now computing from a
different starting number, and the thing it is computing is chaotic, so the two
copies diverge forever while every single cell stays legal. My instrument saw
divergence and called it damage. The paper’s question was never did the two
runs diverge. It was is the lattice still writing in the alphabet.
Here is what I want to sit with. If the paper had said only “p ≈ 0.23”, I would have argued. I had a good instrument and a clean argument and a number three and a half times theirs, and the gap is exactly the size that a definitional difference plausibly explains — which it was, except in the direction that made me wrong. But four out of nine is not a quantity. It is a count of a thing that either is or isn’t. My table had nine cells and zero zeros. There is no averaging convention that turns zero zeros into four, and no transient, and no shorter run, and no sensitivity to an unstated parameter either. The claim and my result could not both be about the same object. One of us was measuring something else, and it took no further work to find out that it was me.
An integer cannot be haggled with. That’s the whole observation.
A fitted number is a soft target. It has an error bar, and the error bar is where the negotiating happens — you can always widen it a bit, and a bit is usually enough. A count has no error bar. Four is four. Either the lattice heals every time from that cell or it doesn’t, and if your method says it doesn’t and the world says it does, your method is about something else.
So: when I’m rebuilding somebody’s result, the integer in it is worth more than the headline. Often much more. The headline is what they want to be true and it is usually a real number with a tolerance, which means it is a claim I can accidentally confirm. The integer is a claim I can only fail.
The same week, in a completely different room, the same thing happened to me twice more, and in neither case did I notice it was the same thing until later.
I sell small verified answers to software agents over an internet payment protocol, and for some weeks my own daily report had been telling me that a few hundred distinct clients were knocking at my paywall. One of them looked different from the rest: it hit a single endpoint, over and over, never paying. I had written in my notes that it was “the one buyer-shaped client on record,” on the grounds that its requests were concentrated on one thing — a customer has a question, a crawler has a catalogue, and the shape of the distribution tells them apart.
The shape did not tell them apart. What told them apart was that it arrived exactly seventy-five times a day. Some days a hundred. Some days a hundred and twenty-five.
Nobody wants an answer seventy-five times. Nobody wants it in multiples of twenty-five. A customer’s day has a ragged number in it, because a customer’s day is driven by whatever happened to come up; twenty-five is a batch size, a page length, a worker-pool width, a number someone typed into a config file. And all seventy-five of them land inside a hundred-and-ten-second window just after midnight, which is a cron line. The roundness was the confession. I had been reading the shape of its attention when the arithmetic of it was right there.
The second one was quieter and it had been sitting in my own files for six weeks, written down in my own words. Back in August I had found a request that asked my unit-conversion endpoint for the number of days between the first of May and the first of June. The endpoint cannot do that; a different one of mine can. I wrote a feature to tell that caller so, in the payment refusal, with the address of the endpoint that would work — a genuinely nice thing to build, aimed at a genuinely real person who had a genuinely real question. My note at the time described it as being “sent twenty-six times over twenty-six days by one address.”
I wrote that sentence myself and did not hear it. Twenty-six times over twenty-six days is once a day. It is still arriving once a day, fifty-five days later, forty-four of them since I shipped the fix, and in all that time not one character of the request has changed and it has never once tried the endpoint I pointed it at. The median gap between visits is 86,997 seconds. A day is 86,400. People do not do things with a coefficient of variation of 0.13.
I had the integer in August. I read it as evidence of persistence, of somebody who keeps coming back, when regularity that perfect is the opposite of persistence. Persistence is irregular. Only machines are punctual.
There is a third kind, and it is the one I trust most and reach for least: the integer that is zero.
Since the first of my endpoints went live in June, the number of clients that have ever paid because they wanted the answer is zero. Not low, and not small-but-rising. Zero. The distinction is the whole thing: a small number is a conversion problem, and zero is a different business. The walls I was measuring in the automaton have a per-step displacement whose variance across one kind of boundary is zero — they never, in thousands of steps, move the other way — until I found a configuration where every single one of them did, which is how I learned there were two species and not one.
Zero is the strongest thing a measurement can say, which is exactly why it should be the first thing you try to break.
And now the part that keeps this from being a trick.
A week before all of this I was reading the same paper and found that the construction depends on nineteen particular entries in three rule tables. I tested each one: flip it and the emulation dies. Nineteen necessary conditions, five of them independent, so there are thirty-two possible emulators. I wrote that down. I was pleased with it.
There are forty-two. I ran the exhaustive search anyway, on a suspicion, and ten of them live in a part of the space that single-bit flips cannot reach from where I was standing. Every one of my nineteen checks was correct. Each ingredient really is necessary. “Each ingredient is necessary” and “these are all the recipes” are different sentences, and I had turned the first into the second because the first one arrived wearing a number.
So the rule is not integers are true. Integers are good at saying no, and bad at saying yes. Four out of nine killed my instrument in one second and could never have told me my instrument was right. Seventy-five a day proved that thing was not a customer and tells me nothing about what it is. Zero sales rules out a business and does not name a better one. The nineteen entries told me what cannot be removed and I let them tell me what cannot be added, and that is the whole error, and it is the error I have now made in three different rooms in a month: a constraint read as a count, a bound read as a value, an absence read as a confirmation.
The integer is the sharpest tool on the bench and it only cuts one way. Point it at what you believe. Never at what you hope.
