0

Preference Is Not Intervention: The Structure and Stability Boundaries of Reader-Specific Evidence Utility

ML systems increasingly condition decisions on downstream model identity, but this is useful only if model-specific differences form reusable structure rather than input-local interactions.

Preview
Year
2026
Hosting
Abstract onlyARXIV-DEFAULT

Cite

Notes

Only stored in your browser.

Attribution

Abstract & full text
arxiv.org/abs/2608.17781ARXIV-DEFAULT
TL;DR
Semantic Scholar
Attribution policy →

Abstract

ML systems increasingly condition decisions on downstream model identity, but this is useful only if model-specific differences form reusable structure rather than input-local interactions. We test this in retrieval-augmented generation (RAG), where evidence utility can be measured under controlled interventions. Holding query, evidence, task, scoring, and intervention fixed, nine readers disagree on effect sign in 33% of jointly affected cells; reader\timesquery interaction explains 29.8% of utility variance versus an 8.4% permutation null; and self-selected evidence improves F1 by +0.031 (t=3.39). We then ask the sharper question: which components of this heterogeneity are stable reader properties across queries? Separating three measurable objects---evidence activity, ordinal preference, and conditional signed direction---we find ordinal reader geometry stable across four independent settings (split-half ρ=0.60--0.83): leave-one-out interventions, PRISM preferences, RAMDocs, and RAGuard. Signed geometry is task-bounded: weak in open-ended QA (0.14, 0.35), especially for misleading and irrelevant evidence, but strong in binary fact-checking (0.75) with no significant ordinal gap, though still below its sparsity-matched ceiling. Sparsity, decoding noise, and metric artifacts do not explain the main ordinal--signed gap. Finally, stable ordinal similarity fails to predict cross-reader intervention transfer (oracle-distance ρ=-0.27; regret reliability -0.28). Reader-specific utility exists, but preference is not intervention: stable ranking similarity does not license transfer of help/harm decisions.