In nonconvex Burer--Monteiro low-rank matrix sensing of U^\ast \in \mathbb{R}^{V \times D} under power-law covariance Σ= diag(j^{-s}){j=1}^V, the precision-wall approximation predicts that Adam stalls when the smallest non-gauge Hessian eigenvalue drops below its denominator floor, giving s^\ast(V,D) = [\log(1/\varepsilon{eff}) - \log V]/\log(VD) + δ_{Adam}. If \varepsilon_{eff} were set by \varepsilon_0, lowering \varepsilon_0 from 10^{-10} to 10^{-12} would raise s^\ast by 0.95--1.33. Instead, across six values of \varepsilon_0 two decades apart, 10^{-2} to 10^{-12}, the boundary follows the \varepsilon_0 slope down to 10^{-6}, bends near 10^{-8}, and is flat below 10^{-8}, while warm-started L-BFGS and Gauss--Newton reach machine precision on stalled instances. We trace this saturation to a self-generated dynamical floor \varepsilon_{dyn} \propto α: high-curvature modes oscillate persistently near the adaptive edge of stability with amplitude \propto α(1-β_1)/(1+β_1) and leak gradient fluctuations of scale κα into rare modes via measurement coupling, pinning \sqrt{\hat{v}V} \approx κα\gg \varepsilon_0. Fitting a single threshold-crossing wall parameter \varepsilon{dyn} = 4.76\times 10^{-9} (with the detection lag frozen at its companion calibration) predicts the saturated failure boundary for D \ge 4 across five (V,D) scales to RMS 0.098 (breaking down at D = 2), while shrinking \varepsilon_0 below 10^{-8} is ineffective and lowering the step size α reduces the floor directly.
A Self-Generated Floor on Adam's $\varepsilon$: Evidence from Power-Law Matrix Sensing
In nonconvex Burer--Monteiro low-rank matrix sensing of $U^\ast \in \mathbb{R}^{V \times D}$ under power-law covariance $Σ= \mathrm{diag}(j^{-s})_{j=1}^V$, the precision-wall approximation predicts that Adam stalls when the smallest non-gauge Hessian eigenvalue drops below its…
- Preview

- Year
- 2026
- Hosting
- Abstract onlyARXIV-DEFAULT
Cite
Notes
Only stored in your browser.
Attribution
- Abstract & full text
- arxiv.org/abs/2611.01868ARXIV-DEFAULT
- TL;DR
- Semantic Scholar