Marginally Useful: Formalizing the Information Gap in Conformal Prediction

Conformal prediction gives finite-sample, distribution-free marginal coverage for a set. The guarantee is real, and it is often misread as evidence of forecast quality. We separate the two with one decomposition, which we call the residual-information gap: for a single-shape residual predictive system, the log-score regret relative to the oracle is exactly the mutual information I(R;X) between the residual and the input. Conformalization re-levels coverage but cannot touch this quantity, because it is a property of the predictor's shape class and not of calibration; no recalibration that ignores X reduces it. The familiar cautions about conformal prediction follow as context: marginal coverage is not conditional, validity is insensitive to sharpness, and the guarantee needs exchangeability.