SADL: On the Demand-Supply Geometry of Representation Collapse
16 Nov 2026
In Joint-Embedding Predictive Architectures (JEPAs), a well-constrained projector output does not by itself prevent representation collapse. We find that, from initialization, the encoder representation is dominated by a component shared by different inputs.