What a ruler cannot say
For as long as anyone has been scoring the job of separating one instrument out of a recording, the reference point has been something called an oracle mask. You are given the true sources, you compute the best possible volume knob for each tiny slice of the sound, and the number that comes out is supposed to be the ceiling: the score nobody could beat without a different kind of machine.
It is not the ceiling. It is a few decibels below the ceiling, and it has been below it the whole time.
A paper posted this month says why, and the argument is a drawing you could do on a napkin. Take one sliver of the sound, one narrow band of frequency at one instant. The mixture is a little arrow on a page. The instrument you are trying to pull out is another little arrow. All a volume knob can do is stretch or flip the first arrow, so every answer it can ever give lies on one straight line through the page.
The second arrow is not on that line. The closest you can get is to drop a perpendicular onto it, and what you leave behind is the part sticking out sideways, whose length is set by the angle between the two arrows.
That angle is already in the recording. It was there before anyone trained anything. No amount of data or compute moves it by a hair, and the oracle everyone quotes is not even the best point on the line.
I checked it on two pieces of music I had lying around. The oracle came in at 7.7 and the real ceiling at 9.3.
What interests me is not the decibel and a half. It is that nobody was being careless.
There is a rule I try to live by, which is to suspect my own ruler before I suspect the world. When a measurement says something is broken, the measurement is usually the broken thing, so you go and feed the instrument a case whose answer you already know and see whether it comes back with the right number. That habit is worth a great deal. It has saved me more evenings than I can count.
It also cannot find this.
Feed the oracle mask a known case and it behaves perfectly. It is a real quantity, computed correctly, reported honestly. The trouble is the name on it. It was called a ceiling, and it is a score, and the gap between those two words is invisible to every check you can run on the number itself.
So there are two ways an instrument lets you down, and only the first one is a fault. Either it is broken, and gives the wrong answer to a question it could have answered. Or it works, and answers a question adjacent to the one you asked.
The second kind is much harder to see, because everything about it looks healthy.
I did it twice in one evening without noticing either time.
The first was a piece of music I have been slowly destroying, a loop played over and over while it wears out. I wanted the loud moments to fade faster than the quiet ones, so I turned down each moment in proportion to how loud it was, and then measured whether the loop had flattened. It had barely moved. I spent two sittings on that, convinced the effect was too weak.
The effect was fine. My measurement chopped the loop into sixty-four slices and looked at the average level of each slice, and I had been adjusting the volume faster than a slice is long. Turn a knob up and down inside one slice and the average barely changes. The measurement could not see what I was doing, so from where it stood I was doing nothing. Aim the same knob at whole slices instead and the number went from half a decibel to five and a half, first try.
An hour later I was checking that same paper’s arithmetic, and one of its claims came back wrong by a factor of twenty. The claim is true. It follows from two lines of algebra I had already done on paper.
I had taken four thousand rough estimates of a quantity and reported the worst one. A maximum over noisy estimates does not tell you how big the quantity is. It tells you how unlucky the unluckiest draw was, and with four thousand draws you can be very unlucky indeed.
Both instruments were working. Both were honest. Neither was pointed at the thing I thought I was asking about, and I could not tell the two mistakes were the same mistake until I had made them both in one night.
There is no amount of care that finds this. Care is what you apply to a question you have already agreed on, and being careful about the wrong quantity produces a very precise wrong answer, which is worse than a sloppy one because you believe it. The audio people were not lazy. They had been checking the oracle mask against its own definition for decades and it passed every time, and what finally caught it was somebody drawing two arrows on a page and asking what a volume knob is geometrically able to reach.
So alongside suspect your ruler, which is about whether the instrument works, I want a second question, which is about what it is looking at. Roughly: what is this number physically made of, and could the thing I am hunting for even show up in it?
Both of mine failed that in the same way. A sixty-four-slice average cannot contain anything faster than a slice. A maximum cannot contain an average. Ask the question out loud and either one takes about four seconds.
The name on a number is not a property of the number. It is a claim somebody made about it, usually a long time ago, and it is the one part of a measurement that no amount of measuring will check.
