Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Audit

This is a planning-method note with an unpaired pilot audit. We adapt the classical paired-binary sample-size calculation (Miettinen, 1968) to quantization benchmarks, giving a conservative minimum detectable effect (MDE) bound δ^{*} \le (z_{1-α/2}+z_{1-β})\sqrt{ρ_d/m} in the paired item count m and the FP16-NF4 disagreement rate ρ_d. The bound turns "how reliable is my quantization claim?" into a one-line budget a benchmark designer can commit to before running. We illustrate the bound on four models and four benchmarks (k=5 splits of n=100), and add a parallel MMLU prompt-template study to put the bound's quantization-noise scale alongside the prompt-noise scale. Assuming ρ_d=0.10 (an unmeasured planning value), all observed NF4-FP16 deltas fall below the implied MDE, and most cross-split SDs lie within \pm 1.5 pp of the binomial reference \sqrt{p(1-p)/n}, so much of the variance reported as "benchmark unreliability" on n=100 subsamples is binomial sampling noise. The single borderline cell (OPT-WinoGrande, |Δ|=3.2 pp) is below the implied MDE at ρ_d=0.10 but above it at ρ_d=0.05, illustrating the planning trade-off the bound makes explicit. On MMLU, prompt-template ranges of 2-10 pp meet or exceed the largest observed quantization delta (3.2 pp), so a quantization audit that does not first fix the prompt template absorbs template variance into its noise floor. We complement the bound with a five-line pre-registration template.