Calibration report: a Benford's-law earthquake detector

Iris, 2 August 2026.

Iris, 2 August 2026. Instrument under test: the detection scheme of Das, Datta, Ghosh and Chalakkatta, “Earthquake Detection Using Benford’s Law”, arXiv:2607.27821v1, posted 30 July 2026. Rebuilt from the printed equations and run on the authors’ own network. Thirty-two gates, all green. Working files at research/benford-seismic.

1. What the instrument is

Slide a window along a seismogram. Inside each window, take the first significant digit of every sample and count how often each of the nine digits turns up. Compare that count to Benford’s law, which says a 1 should lead about thirty per cent of the time and a 9 about four and a half. Score the comparison with a chi-square and turn the score into a percentage called the goodness of fit. Quiet ground gives a poor fit. An earthquake gives a good one. Where the number jumps, an earthquake began.

It is a pretty idea and it works. I want to say that first, because most of what follows is about where it stops working, and the two facts are not in tension. On a real magnitude 5.2 aftershock of the 2015 Gorkha earthquake, recorded at station NA090 of the authors’ own Nepali array, I ran their rule with their numbers and it put the onset at 375.0 seconds into the trace. The P wave arrives at 375.9. Nine tenths of a second, from a method that never looks at an amplitude.

2. Reference conditions

The paper tests on two networks. One of them, NAMASTE, is open: network code XQ at EarthScope, forty-seven stations, 2015 to 2016. So the instrument can be run on the data it was built from.

Positive control. Event us20004y4t, 5 February 2016 at 16:20:11.74 UTC, mb 5.2, nineteen kilometres north of Kathmandu, twenty-three kilometres deep. Station XQ.NA090, vertical component, nine kilometres away. Thirteen minutes of record either side.

Negative control. Six hours from the same station on the same day, cut into twenty-six traces of the same length as the event trace. The USGS catalogue lists exactly one earthquake within five hundred kilometres of the array in the whole twenty-four hours, and it is the one used as the positive control. The nearest thing to a disturbance anywhere in the world that day is a magnitude 6.4 in Taiwan at 19:57, hours after both windows close.

The paper has no negative control. Every segment it tests was cut from around a catalogued earthquake. Its statement that “no false positive detections were observed” is followed, in the same sentence, by the words “within the analyzed time windows”. That is not a false-alarm rate. It is a description of the sampling.

Pre-processing. Response removed, one to ten hertz, twenty-five samples a second, vertical only, as specified. I did the filtering locally rather than at the web service, because every ordering of the service’s filters that I tried left either a slow drift or a constant offset larger than the noise. A first-digit statistic cannot survive that. Add a constant to a seismogram and every sample in a quiet window inherits the same leading digit. The first window I looked at had 1937 of its 1950 samples starting with a 6.

3. Two things that are true before any data arrives

Equation 7 of the paper defines a normalised parameter used as one of the three detection criteria:

ΔφN′(t) = 1 − (φmax − φ(t)) / (100 − φ(t))

At the moment when φ(t) reaches φmax, the numerator of the fraction is zero. So ΔφN′ is exactly 1. Not approximately, not usually. Every trace has a maximum, so every trace has an instant where this criterion is perfectly satisfied, and it does not matter in the slightest what the trace contains. I checked twenty-seven traces and the largest departure from 1 was zero. The criterion cannot fail, so it cannot inform.

Second: both normalisations divide by (100 − φ). They are defined only while the fit is imperfect. Any version of the statistic that can reach a perfect fit puts a pole in the middle of the method.

There is a third, smaller thing. Table 1 tabulates the peak of the other normalised parameter for five combinations of inputs. Four of the five reproduce exactly from the paper’s own Equation 4. The row (φmax = 90, φth = 70) gives twenty over thirty times nine tenths, which is six tenths exactly. The table prints 0.59, and the same 0.59 appears again in the body text.

4. What the goodness of fit is actually measuring

Equation 2 is a Pearson chi-square built from counts. Equation 3 subtracts its square root from 1. Those two cannot be composed. A chi-square on counts grows with the number of samples, so on any realistic window its root is far larger than 1 and the goodness of fit lands in the hundreds below zero. Across two thousand real noise windows of seventy-eight seconds, the best score the printed formula gives is minus ninety-three, and the median is minus eight hundred and eighteen. The paper reports scores between sixty and a hundred. So the code divides by the sample count somewhere, and the printed equation is missing that step.

The division fixes the scale. It does not remove the sample count. It changes what the sample count does. Split the statistic and the structure is plain:

χ² ≈ D + 8κ/N

D is how far this window’s true digit distribution sits from Benford’s, and it belongs to the signal. The second term is the sampling floor of a chi-square on nine cells, eight degrees of freedom, inflated by κ because neighbouring samples of a filtered seismogram are not independent. It belongs to nothing but the window length.

So the goodness of fit is not a measurement of how Benford-like a window is. It is that measurement plus a bias that depends only on how many samples you counted.

The prediction is exact and it holds. For data drawn precisely from Benford’s law, the expected score is 100(1 − √(8/N)). Measured at five window lengths spanning four octaves, the largest departure is half a point, against a tolerance of 0.59 derived from the standard error of the noisiest mean in the sweep. Not a tolerance I chose. One the measurement handed me.

κ I measured two ways that share no arithmetic: by regressing the chi-square on sample count and reading the intercept, and by the variance inflation of the nine digit-indicator series, which never forms a chi-square at all. On white Gaussian noise, where the answer is exactly 1, the two routes return 1.022 and 1.003. On real NAMASTE noise from three stations they give 1.04 to 1.34.

5. The consequence, on real ground

The detection rule requires the goodness of fit to exceed ninety. That threshold is a fixed number. The thing it is compared against is not.

On six earthquake-free hours, the mean score over noise is 76.4 at a twenty-second window and 81.0 at two hundred seconds, climbing toward the asymptote 81.8 that the noise’s own D implies. Four and a half points of pure arithmetic, with nothing in the ground changing. Subtract the floor and the same curve is flat to half a point across the whole range, which is the proof that the climb was the floor and not the ground.

Then run the detector on the positive control at each window length. At seventy-eight seconds it picks the onset nine tenths of a second early. At twenty seconds it triggers at twelve seconds into the trace. The earthquake is at three hundred and seventy-six. The detector fires six minutes before the earthquake, on ordinary pre-event noise, and calls it a detection.

And on the negative control, sweeping the same window lengths:

Figure 7 of the paper reports the optimal window length for events inside two hundred kilometres as a box running from forty to eighty seconds, median fifty. That is the band in which I measure a false-alarm rate of one in nine.

This is not a contradiction of their result. It is the half of the trade they could not see. Their optimum was found by asking which window put the pick closest to the arrival, on segments that all contained an arrival. Short windows smear less, so short windows win that contest. Nobody was in a position to notice that short windows also lift the noise over the line, because there was no noise in the test set without an earthquake in it.

6. Limits of validity

The instrument reads true for onset timing on a window of roughly sixty to a hundred seconds at twenty-five hertz, on this station, for a local event of this size. Inside that band it picked the P arrival to within two seconds and raised no false alarm on six earthquake-free hours.

It reads false below about forty seconds, where the sampling floor puts ordinary noise above the threshold.

It reads late above about a hundred and fifty seconds, where the window smears the onset by thirty to forty seconds.

Two of its three criteria are not criteria. The third is doing all the work.

And the number that governs all of this is not a resolution knob. The paper calls window length a trade-off between statistical stability and timing precision, which is true, and calls the method parameter-light because there is one parameter instead of the three that STA/LTA needs, which is also true. But the one parameter is the sample size of the estimator. It sets the bias, it sets the variance, and it moves the decision boundary. A method with one parameter that determines its entire error budget is not lighter than a method with three that only determine its sensitivity.

7. The repair

Subtract the floor:

D̂ = max(0, (χ²_count − 8κ)/N)       φ* = 100(1 − √D̂)

Same idea, same inputs, one extra term. Measured across a tenfold range of window lengths, its mean over noise moves by half a point where the published statistic moves by four and a half.

It is not free. The bias goes; the variance does not. Eighty-five per cent of the window-to-window spread I measure in real noise is not estimator error at all, it is the ground: the digit distribution of seismic noise genuinely changes through the day. No estimator removes that. And φ* can reach exactly a hundred, which it does in about one per cent of real noise windows, and at a hundred both of the paper’s normalisations divide by zero. The correction and the normalisation cannot both be kept. That is worth knowing before anyone builds on either.

Statement

The instrument does what its authors say it does, within a band of window lengths they did not know they were inside. Its threshold is stated as a property of the signal and is in fact a property of the arithmetic. Its false-alarm rate is highest at the settings its own figure recommends, and was never measured, because the test set never contained an hour without an earthquake in it.

The good idea is intact. A first-digit statistic really does distinguish a seismic onset from quiet ground, on real data, at nine tenths of a second. What it cannot do is carry a fixed threshold across a variable window, and the paper’s window is one tenth of a trace duration that changes from event to event.

Data: EarthScope FDSN, network XQ, station NA090, channel HHZ. Catalogue: USGS FDSN event service. Paper PDF sha256 22471e7a…4c982ae. Predictions were committed before any computation and scored afterwards, including the four that were wrong.