Score-based diffusion models

The Emergence of Reproducibility and Consistency in Diffusion Models

SCORE-BASED GENERATIVE MODELING THROUGH STOCHASTIC DIFFERENTIAL EQUATIONS

Score Approximation, Estimation and Distribution Recovery of Di usion Models on Low-Dimensional Data

Geometric Harmoinic diffusion processes

Training Types

Ordinary Differential Equation (ODE)

Probability Flow Formulation for SDE in Diffusion Processes

Fokker-Planck Equation: This equation describes how the probability density function of the state variable of an SDE evolves over time.

• the CHANGE in density function
• d = dimenstionality of state space, or the space spanned by the process.
  • Each dimension can have own drift term. Each drift term corresponds to the deterministic evolution of the state variable in that particular dimension
• first term corresponds to drift
  • sums each derivative of the deterministic dirft term multiplied by the density function of x with respect to the change in x
• second term corresponds to the diffusion effect or stochastic part
  • sum of each pair of dimensions, we find the derivative of sum of each stochastic processes multiplied by density function, with respect to the two x’s we are looking at

Equation 37 states:

• we are modifying a new drift term f tilde to include stochastic term
• ∇: This denotes the gradient operator, or derivative. ∇x indicates derivative with respect to x. In a multidimensional space, it represents the vector of partial derivatives with respect to each component of x
• ∇⋅[G(x,t)G(x,t)T] represents the divergence of the product of the diffusion term and its transpose, which captures how the spread or dispersion due to randomness affects the drift. G(t) is this is the diffusion term of the SDE
  • EX: if partial deriv is large, means change in x result in large change in dimension. mult by covariance between two variables or dimensions, so if correlated highly, then resulting is matrix valued diffusion term that has strong stochastic influence
  • The divergence applied to this matrix-valued diffusion term helps quantify how the randomness (stochasticity) is spreading or concentrating the probability density in various directions. It's a measure of the 'spreading' effect of the random forces at different points in the state space.
  • We subtract it from new drift term because its a correction that accounts for how stochastic forces influence the overall trend of the system.
  • By doing this, the new drift term balances the deterministic tendencies of the system (as originally captured by f(x,t)) with the additional influence of the random fluctuations. This adjustment ensures that the model accurately reflects both the predictable and unpredictable aspects of the system's dynamics.
  • We care about spread of each stochastic force from each term because it gives us a measure of how strong stochastic processes influence drift term.