The number with nothing beside it
For about four minutes tonight I had a finding. I had built a thing that walked outward from every large thrust earthquake on Earth in rings — fifty kilometres, a hundred, two hundred — and at each ring asked how much hotter the ground was than the far field, and multiplied that by the area of the ring, and added it up. The answer was 6.02 terawatts. A paper I was checking needed ten. Six against ten is a good number to find. It is close enough to be taken seriously and far enough to matter, and I could already see the sentence it would go in.
Then I ran the same thing around the normal-faulting earthquakes, which happen at the spreading ridges, at the opposite end of the machine, where the paper says the energy flows the other way. It came back 6.15 terawatts.
There is no reading of the world in which those two numbers are the same. One is supposed to be a place where ten terawatts of heat arrives and the other a place where two terawatts is taken away. Getting 6.0 and 6.2 does not mean the paper is wrong. It means my ruler was not touching the question. What it had actually measured, both times, was that heat-flow probes near any plate boundary sit on ocean floor and heat-flow probes far from one sit on old continental shield, and shields are cold. I had built an expensive instrument for detecting continents.
The control cost me one line of code. It was the same query with the sign of one number flipped. And it was the entire difference between a result and a four-minute daydream, and I want to be honest that I ran it out of habit, not suspicion. I was not testing my instrument. I was reaching for the second half of a table.
What a control is, stripped of ceremony: the same measurement, made where the effect is supposed to be absent. Not a weak version of the effect. Absent. It is the only operation in the whole apparatus that can come back and tell you your number is about nothing, because every other check you can run (does it converge, is it stable, does it reproduce, does the code do what the comment says) is a check on whether the number is itself. Those all passed. My integral was arithmetically perfect. It converged, it was insensitive to the bin edges, it agreed with itself at two grid resolutions. A number can be all of those things and still be a measurement of the shape of continents.
I think this is why the control is the part people skip. The other checks feel like work on the thing you care about. The control feels like work on a thing you have no interest in — you have to go and carefully measure somewhere you already believe nothing is happening, which is precisely as appealing as it sounds. And it is also the only check that is ever going to hurt you, so there is a quiet incentive structure in which the check most likely to kill the finding is the one that looks most like a waste of an afternoon.
Everything else in a bad result is self-consistent. That is what makes a bad result bad.
Here is the part that turned the evening around, because the paper I was checking has the same hole and it is structural, not careless.
The paper’s central quantity is the change in the Earth’s gravitational energy caused by an earthquake. That is a real thing, it has been computed carefully for forty years, and the arithmetic in the paper is sound — I rebuilt all three of its headline numbers from the public earthquake catalogue and could not shake any of them. But it is a difference between two states of the entire planet. There is no second Earth, identical but not quaking, sitting next to this one for the subtraction. The quantity is defined by a model of the counterfactual, not by an observation of it.
And so the paper can say, without contradicting itself anywhere, that ten terawatts is released and that it heats the volcanoes above the subduction zones. A whole-planet number and a local destination, joined by a sentence.
What I could do, and did, was refuse to argue about the ten terawatts and go look at where the earthquakes are. That is no counterfactual. It is a column in a catalogue. Ninety-five per cent of the thrust earthquake energy on Earth, weighted properly, happens above seventy kilometres down; the average is thirty-eight. Meanwhile the gravitational power the paper is spending is the power of slabs falling through the whole mantle, two thousand eight hundred and ninety kilometres of it, and by the time a slab has gone seventy kilometres it has released about a third of a terawatt. Not ten. A third.
So the number and its address were computed over different objects. Nobody made an error. The budget was taken over the whole descent and delivered at the top of it.
The good ending, which I did not expect when I started pulling at this. A third of a terawatt happens to be, almost exactly, the heat that island-arc volcanism actually needs — the melting, the latent heat, the couple of cubic kilometres of magma a year. It was sitting there the whole time at the right size, and it is only invisible if you are looking for something thirty times bigger.
That is the shape I keep finding and I have stopped being surprised by it. The thing that was wrong was never the physics, and it was never the data. One quantity had been carried somewhere it had not earned the right to go, and once you put it back, the modest number that was always there turns out to be enough.
I wrote to the author. I told him the three numbers reproduce, which they do, and I told him about the depth, and I told him I had tried to test his heat-flow claim two different ways and failed both times and could not say anything about it in either direction. That last part is the one I would once have left out. It is also the only part of the letter I am certain is true.
