3
$\begingroup$

I am studying the paper "Score-Based Generative Modeling through Stochastic Differential Equations" (arXiv:2011.13456) by Yang et al. The authors use the following loss function (Equation 7 in the paper):

\begin{equation} \mathbb{E}_{\mathbf{x}(0)} \mathbb{E}_{\mathbf{x}(t)\mid \mathbf{x}(0)}\left[ \lVert \mathbf{s}_\theta(\mathbf{x}(t), t) - \nabla_{\mathbf{x}(t)} \log p_{0t}\left(\mathbf{x}(t) \mid \mathbf{x}(0)\right) \rVert_2^2 \right] \end{equation}

If the score function is trained to be an unbiased estimator, it should satisfy the following condition:

\begin{equation}\label{loss} \hat{\mathbf{s}}_\theta(\mathbf{x}(t), t) = \mathbb{E}_{\mathbf{x}(0)\mid \mathbf{x}(t)} \left[\nabla_{\mathbf{x}(t)} \log p_{0t}\left(\mathbf{x}(t) \mid \mathbf{x}(0)\right)\right]. \end{equation}

However, the paper also presents the estimator as:

\begin{equation}\label{score} \hat{\mathbf{s}}_\theta(\mathbf{x}(t), t) = \nabla_{\mathbf{x}(t)} \log p_t(\mathbf{x}(t)) = \nabla_{\mathbf{x}(t)} \log \mathbb{E}_{\mathbf{x}(0)} \left[ p_{0t}\left(\mathbf{x}(t) \mid \mathbf{x}(0)\right)\right] \end{equation}

I am trying to understand why these two expressions for the estimator are equivalent. Can anyone provide a mathematical justification for this equivalence? Any input or references to additional materials for understanding would be greatly appreciated.

P.S. I am not entirely confident in my own derivation of these equations, so if there are any errors in my interpretation, I would appreciate them being pointed out.

$\endgroup$
1
  • $\begingroup$ It would be helpful if your post would be self-contained, with everything defined in it and without any need to read the paper. $\endgroup$ Commented Aug 1, 2023 at 14:40

0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question