0

Dead-Direction Signatures: A Cheap Spectral Reading of Singular Complexity

Singular learning theory characterises the complexity of a deep network through the geometry of its loss singularities. The local learning coefficient (LLC), the standard estimator of Watanabe's real log canonical threshold (RLCT, $λ$), reads this geometry as an integrated…

Preview
Year
2026
Hosting
Full text hostedCC-BY-4.0

Cite

Notes

Only stored in your browser.

Attribution

Abstract & full text
arxiv.org/abs/2606.21158CC-BY-4.0
TL;DR
Semantic Scholar
Attribution policy →

Abstract

Singular learning theory characterises the complexity of a deep network through the geometry of its loss singularities. The local learning coefficient (LLC), the standard estimator of Watanabe's real log canonical threshold (RLCT, λ), reads this geometry as an integrated Bayesian scalar through SGLD, which needs per-task calibration and 10^4-10^6 forward-backward passes per checkpoint. We introduce Dead-Direction Signatures (DDS), a family of cheap closed-form spectral readings of singular structure: each reads a network's activation matrix or per-sample-gradient Fisher-Gram at a chosen layer, replacing the SGLD posterior chain with spectral linear algebra. The readings rest on a dead-direction framework that predicts a structural correlation between activation- and Fisher-side spectra at any singular minimum, and a rank-multiplicative volume identity that single-eigenvalue monitors cannot produce: the active-volume \log\det^{+}(G) slope counts the dead directions, tracking the rank-deficit r across r \in {1,2,3,4} (slope ratios 2.0, 3.1, 4.0 at r{=}2,3,4 against the predicted 2,3,4), where the smallest eigenvalue is rank-blind. On reduced-rank regression with closed-form λ, calibrated LLC recovers λ at 99% mean and the DDS observables rank-track it at the framework-predicted sign; on a non-linear modular-addition transformer DDS separates d_{model} across eighteen orders of magnitude where calibrated LLC at the protocol budget is rank-flat. Complementary to LLC's integrated posterior reading, DDS gives a directional, layer-local handle on a network's dead directions, read in closed form from its activation and gradient spectra.