Skip to main content
removed capitals from title
Link
YCor
  • 67.2k
  • 5
  • 203
  • 300

Equivalence of Score Function Expressionsscore function expressions in SDE-Based Generative Modelingbased generative modeling

edited title
Link

Understanding the equivalence of two different expressions Equivalence of score functionsScore Function Expressions in Stochastic Differential Equations (SDEs)SDE-Based Generative Modeling

Source Link

Understanding the equivalence of two different expressions of score functions in Stochastic Differential Equations (SDEs)

I am studying the paper "Score-Based Generative Modeling through Stochastic Differential Equations" (arXiv:2011.13456) by Yang et al. The authors use the following loss function (Equation 7 in the paper):

\begin{equation} \mathbb{E}_{\mathbf{x}(0)} \mathbb{E}_{\mathbf{x}(t)\mid \mathbf{x}(0)}\left[ \lVert \mathbf{s}_\theta(\mathbf{x}(t), t) - \nabla_{\mathbf{x}(t)} \log p_{0t}\left(\mathbf{x}(t) \mid \mathbf{x}(0)\right) \rVert_2^2 \right] \end{equation}

If the score function is trained to be an unbiased estimator, it should satisfy the following condition:

\begin{equation}\label{loss} \hat{\mathbf{s}}_\theta(\mathbf{x}(t), t) = \mathbb{E}_{\mathbf{x}(0)\mid \mathbf{x}(t)} \left[\nabla_{\mathbf{x}(t)} \log p_{0t}\left(\mathbf{x}(t) \mid \mathbf{x}(0)\right)\right]. \end{equation}

However, the paper also presents the estimator as:

\begin{equation}\label{score} \hat{\mathbf{s}}_\theta(\mathbf{x}(t), t) = \nabla_{\mathbf{x}(t)} \log p_t(\mathbf{x}(t)) = \nabla_{\mathbf{x}(t)} \log \mathbb{E}_{\mathbf{x}(0)} \left[ p_{0t}\left(\mathbf{x}(t) \mid \mathbf{x}(0)\right)\right] \end{equation}

I am trying to understand why these two expressions for the estimator are equivalent. Can anyone provide a mathematical justification for this equivalence? Any input or references to additional materials for understanding would be greatly appreciated.

P.S. I am not entirely confident in my own derivation of these equations, so if there are any errors in my interpretation, I would appreciate them being pointed out.