A stochastic subgradient method with optimal failure exponent

Fix a target accuracy \varepsilon, a gradient-noise level s, and a horizon N. We wish to design algorithms which minimize the probability of observing a suboptimality gap which exceeds the target accuracy, i.e., E_N = -\log \sup_{f,P} \mathbb{P}P(f(x_A) - f\star \ge \varepsilon), with the noise law P known only to be sub-Gaussian. We single out a uniformly averaged schedule which is harmonic (of the form h_k = R^2/(\varepsilon (N+m-k))) and prove, via an optimized exponential supermartingale argument, that it attains the optimal exponent E_N^\star = \varepsilon^2 N (1+o(1))/(2R^2 s^2). Optimality is certified by a matching impossibility result: under Gaussian noise, a gradient-masking change of measure caps the exponent of every algorithm at the same leading order. In a small-noise limit, our setting degenerates into the adversarial-error model of Gösgens and van Parys (2025) for subgradient methods.