Optimal score function estimation via derivatives constraints

We consider the problem of score function estimation via empirical risk minimization. We first start with the question of inferring the score function of a probability measure μ with density on the flat torus from a sample of distribution μ. We show that constraining the hypothesis space to a Sobolev ball is sufficient to prevent overfitting and obtaining minimax estimation rates. We then consider the problem of score function estimation in the context of score-based generative modeling. Again, under a conjecture tying the score estimation rates to the quality of the output of a score-based generative model, we obtain minimax rates for such an approach using score function estimators obtained by constraining the hypothesis class to a Sobolev ball.