Best-of-$N$ TTS Evaluation is Confounded by ASR Family Alignment

Best-of-N (BoN) inference improves content consistency in zero-shot text-to-speech by selecting from N candidates with an automatic speech recognition (ASR) verifier. We identify an underexplored evaluation confound: a verifier's apparent quality depends strongly on which ASR family judges it. On LibriSpeech-PC test-clean \citep{librispeechpc} with F5-TTS \citep{f5tts}, verifier rankings reverse across Whisper, wav2vec 2.0, and HuBERT evaluators, and same-family verifier-evaluator pairs recover 2-3\times more oracle headroom than cross-family pairs despite near-identical representations (linear CKA 0.978) -- a pattern consistent with identity- or lineage-level coupling rather than representational overlap. We propose two cross-family rank ensembles (rank-averaging and conjunctive max-rank) that attain the lowest mean WER across three independent evaluators -- 1.61% at N{=}10 (-12% relative to F5-TTS) -- with no measurable degradation under automatic SIM-o/UTMOS metrics; the best single verifier drives WER from 2.06% to 1.72% (-16.5%) under the official F5-TTS evaluator. We recommend cross-evaluator triangulation as default reporting practice.