Introduction

生成对象（Object）：对图像，视频，蛋白质等数据类型可视为向量，即 $z\in {\mathbb{R}}^d$

生成（Generation）：从数据分布中采样，$z\sim p_{data}$

数据集（Dataset）：服从数据分布的有限样本，$z_1,...,z_N\sim p_{data}$

条件生成（Conditional Generation）：从条件分布中采样，$z\sim p_{data}(\cdot \mid y)$

目标：训练生成模型，将初始分布（$p_{\text{init}}$）的样本转化为数据分布样本$p_{\text{data}}$

Flow and Diffusion Models

通过模拟常微分方程（Ordinary Differential Equations, ODEs）和随机微分方程（Stochastic Differential Equations, SDEs）可以实现从初始分布到数据分布的转换，分别对应Flow Model和Diffusion Model

Flow Models

Flow Model可以由ODE来描述，即
$$\begin{gathered} X_0\sim p_{\text{init}}▹\text{random init} \\ \frac{d}{dt}{X}_t=u_t^{\theta }(X_t)▹\text{ODE} \\ \text{Goal: }{X}_1\sim p_{\text{data}}\iff {\psi }_1^{\theta }(X_0)\sim p_{\text{data}} \end{gathered}$$
其中向量场 $u_t^{\theta }:{\mathbb{R}}^d\times [0,1]\to {\mathbb{R}}^d$ 为神经网络，$\theta$为参数。${\psi }_t^{\theta }$描述了由$u_t^{\theta }$引起的Flow，为ODE方程解（Trajectory）的集合

通过使用Euler算法，可以模拟ODE计算出Flow，实现从Flow Model中采样

Diffusion Models

Diffusion Model可以由SDEs描述，如下所示（由于其随机性SDEs不使用微分表示形式）
$$\begin{gathered} d{X}_t=u_t^{\theta }(X_t)dt+{\sigma }_{t}d{W}_t▹\text{SDE} \\ {X}_0\sim p_{init}▹\text{random initialization} \\ \text{Goal: }{X}_1\sim p_{\text{data}} \end{gathered}$$
其中 ${\sigma }_t\ge 0$为diffusion系数，$W_t$为随机过程 布朗运动（Brownian motion）

可以看出Diffusion Model是Flow Model的一个拓展，当${\sigma }_t=0$时即为Flow Model

同样的，可以使用以下算法实现从Diffusion Model中采样

Training Target and Train Loss

对于Flow Model和Diffusion Model
$$\begin{array}{rlr}{X}_0\sim p_{\text{init}},d{X}_t& =u_t^{\theta }(X_t)dt& \text{(Flow model)}\\ X_0\sim p_{\text{init}},d{X}_t& =u_t^{\theta }(X_t)dt+{\sigma }_{t}d{W}_t& \text{(Diffusion model)}\end{array}$$

训练可以通过最小化以下损失实现
$$\mathcal{L}(\theta )={\Vert u_t^{\theta }(x)-\underset{\text{training target}}{\underbrace{u_t^{\text{target}}(x)}}\Vert }^2$$

$u_t^{\theta }$ 为网络模型，$u_t^{\text{target}}(x)$为目标向量场，其实现将初始数据分布转化为目标数据分布，为了实现计算 $\mathcal{L}(\theta )$ 或者间接计算 $\mathcal{L}(\theta )$需要构建$u_t^{\text{target}}(x)$。

Probability Path

Probability Path是从初始分布到目标数据分布的渐进插值（gradual interpolation），分为条件概率路径（conditional probability path）和边缘概率路径（marginal probability path），分别为$p_t(\cdot \mid z)$ 和 $p_t(\cdot )$，其中：
$$p_0(\cdot \mid z)=p_{\text{init}},p_1(\cdot \mid z)={\delta }_z\text{for all }z\in {\mathbb{R}}^d$$
$p_t(\cdot )$ 可由以下公式获得
$$\begin{array}{rlr}& z\sim p_{\text{data}},x\sim p_t(\cdot \mid z)⟹x\sim p_t& ▹\text{sampling from marginal path}\\ & p_t(x)=\int p_t(x\mid z)p_{\text{data}}(z)dz& ▹\text{density of marginal path}\\ & p_0=p_{\text{init}}\text{and}{p}_1=p_{\text{data}}& ▹\text{noise-data interpolation}\end{array}$$

Training Target for Flow Model

对于$z\in {\mathbb{R}}^d\sim p_{data}$，记$u_t^{target}(\cdot \mid z)$为条件概率路径 $p_t(\cdot \mid z)$ 对应的条件向量场，即
$$X_0\sim p_{\text{init}},\frac{\mathrm{d}}{\mathrm{d}t}{X}_t=u_t^{\text{target}}(X_t|z)\Rightarrow X_t\sim p_t(\cdot |z)(0\le t\le 1)$$
则$u_t^{target}(x)$可定义为
$$u_t^{\text{target}}(x)=\int u_t^{\text{target}}(x|z)\frac{p_t(x|z)p_{\text{data}}(z)}{p_t(x)}\mathrm{d}z$$
且满足：
$$X_0\sim p_{\text{init}},\frac{\mathrm{d}}{\mathrm{d}t}{X}_t=u_t^{\text{target}}(X_t)\Rightarrow X_t\sim p_t(0\le t\le 1)$$
其中$X_1\sim p_{data}$。

这可以由Continuity Equation 证明

Continuity Equation

对于向量场$u_t^{target}$ 且 $X_0\sim p_{init}$，有$X_t\sim p_t$ 在$0\le t\le 1$ 成立有且仅有
$${\partial }_t{p}_t(x)=-\mathrm{div}(p_t{u}_t^{\text{target}})(x)\text{for all }x\in {\mathbb{R}}^d,0\le t\le 1$$
其中${\partial }_t{p}_t(x)=\frac{\mathrm{d}}{\mathrm{d}t}{p}_t(x)$，$\mathrm{div}(v_t)(x)={\sum }_{i=1}^d\frac{\partial }{\partial x_i}{v}_t(x)$

Training Target for Diffusion Model

同样的，对于Diffusion Model，可以构建$u_t^{target}$如下所示，满足$X_t\sim p_t(0\le t\le 1)$ ，即
$$\begin{array}{rl}& X_0\sim p_{\text{init}},\mathrm{d}{X}_t=\left[u_t^{\text{target}}(X_t)+\frac{{\sigma }_t^2}{2}\nabla \log p_t(X_t)\right]\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t\\ & \Rightarrow X_t\sim p_t(0\le t\le 1)\end{array}$$
并且将$p_t(x),u_t^{target}$ 替换为 $p_t(x\mid z),u_t^{target}(x\mid z)$ 时仍然成立

其中，$\nabla \log p_t(x)$ 称为marginal score function，$\nabla \log p_t(x\mid z)$ 称为conditional score function，二者满足
$$\nabla \log p_t(x)=\frac{\nabla p_t(x)}{p_t(x)}=\frac{\nabla \int p_t(x|z)p_{\text{data}}(z)\mathrm{d}z}{p_t(x)}=\frac{\int \nabla p_t(x|z)p_{\text{data}}(z)\mathrm{d}z}{p_t(x)}=\int \nabla \log p_t(x|z)\frac{p_t(x|z)p_{\text{data}}(z)}{p_t(x)}\mathrm{d}z$$
这可以由Fokker-Planck Equation证明

Fokker-Planck Equation

对于$X_0\sim p_{\text{init}},\mathrm{d}{X}_t=u_t(X_t)\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t$ 描述的SDE，$X_t\sim p_t$ 成立，当且仅当
$${\partial }_t{p}_t(x)=-\mathrm{div}(p_t{u}_t)(x)+\frac{{\sigma }_t^2}{2}\Delta p_t(x)\text{for all }x\in {\mathbb{R}}^d,0\le t\le 1$$
其中，$\Delta w_t(x)={\sum }_{i=1}^d\frac{{\partial }^2}{\partial x_i^2}{w}_t(x)=\mathrm{div}(\nabla w_t)(x)$

Remark Langevin dynamics

当$p_t=p$时，即概率路径为静态时，有
$$\mathrm{d}{X}_t=\frac{{\sigma }_t^2}{2}\nabla \log p(X_t)\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t$$
此时 $X_0\sim p\Rightarrow X_t\sim p(t\ge 0)$，即Langevin dynamics

Gaussian probability path

设噪声调度${\alpha }_t,{\beta }_t$为单调连续可微函数且${\alpha }_0={\beta }_1=0,{\alpha }_1={\beta }_0=1$，定义Gaussian conditional probability path为
$$p_t(\cdot |z)=\mathcal{N}({\alpha }_{t}z,{\beta }_t^2{I}_d)$$
其满足 $p_0(\cdot |z)=\mathcal{N}({\alpha }_{0}z,{\beta }_0^2{I}_d)=\mathcal{N}(0,I_d),\text{and}{p}_1(\cdot |z)=\mathcal{N}({\alpha }_{1}z,{\beta }_1^2{I}_d)={\delta }_z$

则从其marginal path中采样可以通过以下方法得到
$$z\sim p_{\text{data}},ϵ\sim p_{\text{init}}=\mathcal{N}(0,I_d)\Rightarrow x={\alpha }_{t}z+{\beta }_tϵ\sim p_t$$
基于Gaussian probability path的conditional Gaussian vector field可以计算得到
$$u_t^{\text{target}}(x|z)=\left({\dot{\alpha }}_t-\frac{{\dot{\beta }}_t}{{\beta }_t}{\alpha }_t\right)z+\frac{{\dot{\beta }}_t}{{\beta }_t}x$$
其中${\dot{\alpha }}_t={\partial }_t{\alpha }_t$，${\dot{\beta }}_t={\partial }_t{\beta }_t$

同样的可以得到其marginal score function为
$$\nabla \log p_t(x|z)=-\frac{x-{\alpha }_{t}z}{{\beta }_t^2}$$

Flow Matching

对于Flow Model，定义flow matching loss为
$$\begin{array}{rl}{\mathcal{L}}_{\text{FM}}(\theta )& ={\mathbb{E}}_{t\sim \text{Unif},x\sim p_t}[\Vert u_t^{\theta }(x)-u_t^{\text{target}}(x){\Vert }^2]\\ & ={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},x\sim p_t(\cdot |z)}[\Vert u_t^{\theta }(x)-u_t^{\text{target}}(x){\Vert }^2]\end{array}$$

$z\sim p_{\text{data}},x\sim p_t(\cdot \mid z)⟹x\sim p_t$

定义conditional flow matching loss为

$${\mathcal{L}}_{\text{CFM}}(\theta )={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},x\sim p_t(\cdot |z)}[\Vert u_t^{\theta }(x)-u_t^{\text{target}}(x|z){\Vert }^2]$$
其中$u_t^{\text{target}}(x|z)$可以人为构造获得（例如Gaussian probability path）

可以证明，
$${\mathcal{L}}_{\text{FM}}(\theta )={\mathcal{L}}_{\text{CFM}}(\theta )+C$$
即
$${\nabla }_{\theta }{\mathcal{L}}_{\text{FM}}(\theta )={\nabla }_{\theta }{\mathcal{L}}_{\text{CFM}}(\theta )$$
因此优化${\mathcal{L}}_{\text{CFM}}$即优化${\mathcal{L}}_{\text{FM}}$，而对于${\mathcal{L}}_{\text{CFM}}$，只需构造probability path即可，至此可以得到训练Flow Model的算法，整个流程即称为Flow Matching

Flow Matching for Gaussian Conditional Probability Paths

对于Gaussian Probability Path，有
$$ϵ\sim \mathcal{N}(0,I_d)\Rightarrow x_t={\alpha }_{t}z+{\beta }_tϵ\sim \mathcal{N}({\alpha }_{t}z,{\beta }_t^2{I}_d)=p_t(\cdot |z)$$

$$u_t^{\mathrm{target}}(x|z)=\left({\dot{\alpha }}_t-\frac{{\dot{\beta }}_t}{{\beta }_t}{\alpha }_t\right)z+\frac{{\dot{\beta }}_t}{{\beta }_t}x$$

$$\begin{array}{c}{\mathcal{L}}_{\mathrm{CFM}}(\theta )={\mathbb{E}}_{t\sim \mathrm{Unif},z\sim p_{\mathrm{data}},x\sim \mathcal{N}({\alpha }_{t}z,{\beta }_t^2{I}_d)}[\Vert u_t^{\theta }(x)-\left({\dot{\alpha }}_t-\frac{{\dot{\beta }}_t}{{\beta }_t}{\alpha }_t\right)z-\frac{{\dot{\beta }}_t}{{\beta }_t}x{\Vert }^2]\\ \stackrel{(i)}{=}{\mathbb{E}}_{t\sim \mathrm{Unif},z\sim p_{\mathrm{data}},ϵ\sim \mathcal{N}(0,I_d)}[\Vert u_t^{\theta }({\alpha }_{t}z+{\beta }_tϵ)-({\dot{\alpha }}_{t}z+{\dot{\beta }}_tϵ){\Vert }^2]\end{array}$$

特别的，对于${\alpha }_t=t$，${\beta }_t=1-t$，有
$$p_t(x|z)=\mathcal{N}(tz,(1-t{)}^2)$$

$${\mathcal{L}}_{\mathrm{cfm}}(\theta )={\mathbb{E}}_{t\sim \mathrm{Unif},z\sim p_{\mathrm{data}},ϵ\sim \mathcal{N}(0,I_d)}[\Vert u_t^{\theta }(tz+(1-t)ϵ)-(z-ϵ){\Vert }^2]$$

称之为(Gaussian) CondOT probability path，训练过程如下所示

Score Matching

对于Diffusion Models，由于$u_t^{target}$ 难以得到，因此使用score network ${\sigma }_t^2:{\mathbb{R}}^d\times [0,1]\to \mathbb{R}$对score function进行拟合，同样的，存在score matching loss和conditional score matching loss如下
$$\begin{array}{rl}{\mathcal{L}}_{\text{SM}}(\theta )& ={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},x\sim p_t(\cdot |z)}[\Vert s_t^{\theta }(x)-\nabla \log p_t(x){\Vert }^2]▹\text{ score matching loss}\\ {\mathcal{L}}_{\text{CSM}}(\theta )& ={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},x\sim p_t(\cdot |z)}[\Vert s_t^{\theta }(x)-\nabla \log p_t(x|z){\Vert }^2]▹\text{ conditional score matching loss}\end{array}$$
同样的，虽然$\nabla \log p_t(x)$未知，但$\nabla \log p_t(x\mid z)$可以人工构造，且存在
$$\begin{array}{rl}& {\mathcal{L}}_{\text{SM}}(\theta )={\mathcal{L}}_{\text{SFM}}(\theta )+C\\ & ⟹{\nabla }_{\theta }{\mathcal{L}}_{\text{SM}}(\theta )={\nabla }_{\theta }{\mathcal{L}}_{\text{CSM}}(\theta )\end{array}$$
因此，优化${\mathcal{L}}_{\text{CSM}}(\theta )$即可，此时采样过程如下所示
$$X_0\sim p_{\text{init}},\mathrm{d}{X}_t=\left[u_t^{\theta }(X_t)+\frac{{\sigma }_t^2}{2}{s}_t^{\theta }(X_t)\right]\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t⟹X_1\sim p_{data}$$
其中，尽管理论上对任意${\sigma }_t\ge 0$均可实现采样，但由于存在对随机微分方程模拟不精确导致的精度误差，以及训练误差，因此存在一个最优的${\sigma }_t$。同时观察采样过程可以发现模拟该SDE还需学习$u_t^{\theta }$，但其实通常可以使用一个两输出的网络同时处理$u_t^{\theta }$和$s_t^{\theta }$，并且对于特定的概率路径，两者可以相互转化。

Denoising Diffusion Models: Score Matching for Gaussian Probability Paths

对于Gaussian Probability Paths，有
$$\nabla \log p_t(x|z)=-\frac{x-{\alpha }_{t}z}{{\beta }_t^2}$$
则
$$\begin{array}{rl}{\mathcal{L}}_{\text{CSM}}(\theta )& ={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},x\sim p_t(\cdot |z)}\left[{\Vert s_t^{\theta }(x)+\frac{x-{\alpha }_{t}z}{{\beta }_t^2}\Vert }^2\right]\\ & ={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},ϵ\sim \mathcal{N}(0,I_d)}\left[{\Vert s_t^{\theta }({\alpha }_{t}z+{\beta }_tϵ)+\frac{ϵ}{{\beta }_t}\Vert }^2\right]\\ & ={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},ϵ\sim \mathcal{N}(0,I_d)}\left[\frac{1}{{\beta }_t^2}{\Vert {\beta }_t{s}_t^{\theta }({\alpha }_{t}z+{\beta }_tϵ)+ϵ\Vert }^2\right]\end{array}$$
由于$\frac{1}{{\beta }_t^2}$在${\beta }_t$趋近于0时loss趋近于无穷大，因此通常舍弃常数项$\frac{1}{{\beta }_t^2}$，并用以下方法reparameterize $s_t^{\theta }$为${ϵ}_t^{\theta }$（噪声预测网络）得到DDPM损失函数
$$-{\beta }_t{s}_t^{\theta }(x)={ϵ}_t^{\theta }(x)\Rightarrow {\mathcal{L}}_{\text{DDPM}}(\theta )={\mathbb{E}}_{t\sim \text{Unif},z\sim p_{\text{data}},ϵ\sim \mathcal{N}(0,I_d)}\left[{\Vert {ϵ}_t^{\theta }({\alpha }_{t}z+{\beta }_tϵ)-ϵ\Vert }^2\right]$$
其训练过程如下所示

此外，对于Gaussian Probability Paths，vector field和score可以相互转化，即
$$\begin{gathered} u_t^{\text{target}}(x|z)=\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t\right)\nabla \log p_t(x|z)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x \\ {u}_t^{\text{target}}(x)=\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t\right)\nabla \log p_t(x)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x \end{gathered}$$

proof
$$u_t^{\text{target}}(x|z)=\left({\dot{\alpha }}_t-\frac{{\dot{\beta }}_t}{{\beta }_t}{\alpha }_t\right)z+\frac{{\dot{\beta }}_t}{{\beta }_t}x\stackrel{(i)}{=}\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t\right)\left(\frac{{\alpha }_{t}z-x}{{\beta }_t^2}\right)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x=\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t\right)\nabla \log p_t(x|z)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x$$

$$\begin{array}{rl}{u}_t^{\text{target}}(x)& =\int u_t^{\text{target}}(x|z)\frac{p_t(x|z)p_{\text{data}}(z)}{p_t(x)}\mathrm{d}z\\ & =\int \left[\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t\right)\nabla \log p_t(x|z)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x\right]\frac{p_t(x|z)p_{\text{data}}(z)}{p_t(x)}\mathrm{d}z\\ & \stackrel{(i)}{=}\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t\right)\nabla \log p_t(x)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x\end{array}$$

$u_t^{\theta }$和$s_t^{\theta }$ 也可以相互转化，有
$$u_t^{\theta }(x)=\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t\right)s_t^{\theta }(x)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x$$

$$s_t^{\theta }(x)=\frac{{\alpha }_t{u}_t^{\theta }(x)-{\dot{\alpha }}_{t}x}{{\beta }_t^2{\alpha }_t-{\alpha }_t{\dot{\beta }}_t{\beta }_t}$$

因此对于Gaussian probability paths来说，只需训练$u_t^{\theta }$或$s_t^{\theta }$ 即可，且使用flow matching或者使用score matching的方法均可

最后，对于训练好的$s_t^{\theta }$ 从SDE中采样过程如下
$$\begin{gathered} X_0\sim p_{\text{init}},\mathrm{d}{X}_t=\left[\left({\beta }_t^2\frac{{\dot{\alpha }}_t}{{\alpha }_t}-{\dot{\beta }}_t{\beta }_t+\frac{{\sigma }_t^2}{2}\right)s_t^{\theta }(x)+\frac{{\dot{\alpha }}_t}{{\alpha }_t}x\right]\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t \\ ⟹X_1=p_{data} \end{gathered}$$

Summary

总的来说，Flow Matching比Score Matching更简洁并且Flow Matching更具有拓展性，可以实现从一个任意初始分布$p_{init}$得到任意分布$p_{data}$，但是denoising diffusion models只适用于Gaussian initial distributions and Gaussian probability path。Flow Matching类似于Stochastic Interpolants。

Conditional (Guided) Generation

在给定条件下进行生成（generate an object conditioned on some additional information），称之为conditional generation，为了和conditional vector field区分多称为guided generation

用数学语言描述即，对于$y\in \mathcal{Y}$，对$p_{data}(x\mid y)$中采样，因此模型包含条件向量场$u_t^{\theta }(\cdot \mid y)$，模型架构如下所示
$$\begin{array}{rl}\text{Neural network: }& u_t^{\theta }:{\mathbb{R}}^d\times \mathcal{Y}\times [0,1]\to {\mathbb{R}}^d,(x,y,t)\mapsto u_t^{\theta }(x|y)\\ \text{Fixed: }& {\sigma }_t:[0,1]\to [0,\infty ),t\mapsto {\sigma }_t\end{array}$$
对于给定的$y\in {\mathbb{R}}^{d_y}$，采样过程可以描述为
$$\begin{array}{rlr}\text{Initialization:}& X_0\sim p_{\text{init}}& ▹\text{ Initialize with simple distribution}\\ \text{Simulation:}& \mathrm{d}{X}_t=u_t^{\theta }(X_t|y)\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t& ▹\text{ Simulate SDE from }t=0\text{ to }t=1.\\ \text{Goal:}& X_1\sim p_{\text{data}}(\cdot |y)& ▹X_1\text{ to be distributed like }{p}_{\text{data}}(\cdot |y)\end{array}$$
上述在${\sigma }_t=0$时即为guided flow model

Guided Models

Guided Flow Models的训练损失（优化目标，或者说guided conditional flow matching objective）很容的得到，如下所示
$$\begin{array}{rl}{\mathcal{L}}_{\text{CFM}}^{\text{guided}}(\theta )& ={\mathbb{E}}_{(z,y)\sim p_{\text{data}}(z,y),t\sim \text{Unif}(0,1),x\sim p_t(\cdot |z)}\left[{\Vert u_t^{\theta }(x|y)-u_t^{\text{target}}(x|z)\Vert }^2\right]\end{array}$$
同样的，对于Guided Diffusion Models，有guided conditional score matching objective如下
$$\begin{array}{rl}{\mathcal{L}}_{\text{CSM}}^{\text{guided}}(\theta )& ={\mathbb{E}}_{\square }\left[\Vert s_t^{\theta }(x|y)-\nabla \log p_t(x|z){\Vert }^2\right]\\ \square & =(z,y)\sim p_{\text{data}}(z,y),t\sim \text{Unif}(0,1),x\sim p_t(\cdot |z)\end{array}$$
虽然理论上上述以及足够生成标签$y$对应样本，但是实际上生成效果并不十分fit $y$，以及，无法控制生成内容对label的fit程度。一种解决方法是人为加强$y$的作用，比较先进的技术是Classifier-Free Guidance。

Classifier-Free Guidance

对于Flow Models，以Gaussian probability paths为例
$$\begin{array}{r}{u}_t^{\text{target}}(x|y)=a_{t}x+b_t\nabla \log p_t(x|y)\end{array}$$
其中
$$\begin{array}{r}(a_t,b_t)=\left(\frac{{\dot{\alpha }}_t}{{\alpha }_t},\frac{{\dot{\alpha }}_t{\beta }_t^2-{\dot{\beta }}_t{\beta }_t{\alpha }_t}{{\alpha }_t}\right)\end{array}$$
又
$$\begin{array}{r}\nabla \log p_t(x|y)=\nabla \log \left(\frac{p_t(x)p_t(y|x)}{p_t(y)}\right)=\nabla \log p_t(x)+\nabla \log p_t(y|x)\end{array}$$
则
$$\begin{array}{r}{u}_t^{\text{target}}(x|y)=a_{t}x+b_t(\nabla \log p_t(x)+\nabla \log p_t(y|x))=u_t^{\text{target}}(x)+b_t\nabla \log p_t(y|x)\end{array}$$
可以看出，guided vector field是由unguided vector field和guided score相加得到，一种很自然的想法是对guided score进行加权，得到
$$\begin{array}{r}{\tilde{u}}_t(x|y)=u_t^{\text{target}}(x)+w{b}_t\nabla \log p_t(y|x)\end{array}$$
其中guided score可以看作是噪声类别分类器，早期的工作确实使用这样的方法实现，但是进一步对guided score进行分析得到如下：
$$\begin{array}{rl}{\tilde{u}}_t(x|y)& =u_t^{\text{target}}(x)+w_b\nabla \log p_t(y|x)\\ & =u_t^{\text{target}}(x)+w_b(\nabla \log p_t(x|y)-\nabla \log p_t(x))\\ & =u_t^{\text{target}}(x)-(w_{a}x+w_b\nabla \log p_t(x))+(w_{a}x+w_b\nabla \log p_t(x|y))\\ & =(1-w)u_t^{\text{target}}(x)+w{u}_t^{\text{target}}(x|y).\end{array}$$
即${\tilde{u}}_t(x|y)$由unguided vector field和guided vector field加权得到，并且，通过构造$y=\varnothing$其对应概率为人为设计的超参数$\eta$，从而实现使用$u_t^{\text{target}}(x|\varnothing )$代替$u_t^{\text{target}}(x)$，具体可公式化描述为
$$\begin{array}{rl}{\mathcal{L}}_{\text{CFM}}^{\text{CFG}}(\theta )& ={\mathbb{E}}_{\square }\left[\Vert u_t^{\theta }(x|y)-u_t^{\text{target}}(x|z){\Vert }^2\right]\\ \square & =(z,y)\sim p_{\text{data}}(z,y),t\sim \text{Unif}(0,1),x\sim p_t(\cdot |z),\text{replace }y=\varnothing \text{ with prob. }\eta \end{array}$$

对于Diffusion Models，${\tilde{s}}_t(x|y)$同样可改写如下
$$\begin{array}{rl}{\tilde{s}}_t(x|y)& =\nabla \log p_t(x)+w\nabla \log p_t(y|x)\\ & =\nabla \log p_t(x)+w(\nabla \log p_t(x|y)-\nabla \log p_t(x))\\ & =(1-w)\nabla \log p_t(x)+w\nabla \log p_t(x|y)\\ & =(1-w)\nabla \log p_t(x|\varnothing )+w\nabla \log p_t(x|y)\end{array}$$
training objective如下
$$\begin{array}{rl}{\mathcal{L}}_{\text{CSM}}^{\text{CFG}}(\theta )& ={\mathbb{E}}_{\square }\left[\Vert s_t^{\theta }(x|(1-\xi )y+\xi \varnothing )-\nabla \log p_t(x|z){\Vert }^2\right]\\ \square & =(z,y)\sim p_{\text{data}}(z,y),t\sim \text{Unif}(0,1),x\sim p_t(\cdot |z),\text{replace }y=\varnothing \text{ with prob. }\eta \end{array}$$
训练时，我们通常也可同时优化$s_t^{\theta }(x|y)$和$u_t^{\theta }(x|y)$，对应的，有
$$\begin{array}{rl}{\tilde{s}}_t^{\theta }(x|y)& =(1-w)s_t^{\theta }(x|\varnothing )+w{s}_t^{\theta }(x|y),\\ {\tilde{u}}_t^{\theta }(x|y)& =(1-w)u_t^{\theta }(x|\varnothing )+w{u}_t^{\theta }(x|y).\end{array}$$

采样时，有
$$\mathrm{d}{X}_t=\left[{\tilde{u}}_t^{\theta }(X_t|y)+\frac{{\sigma }_t^2}{2}{s}_t^{\theta }(X_t|y)\right]\mathrm{d}t+{\sigma }_t\mathrm{d}{W}_t$$

Network architectures

网络模型的设计随建模数据的复杂程度各有差别，但都需满足
$$\text{Neural network: }{u}_t^{\theta }:{\mathbb{R}}^d\times \mathcal{Y}\times [0,1]\to {\mathbb{R}}^d,(x,y,t)\mapsto u_t^{\theta }(x|y)$$

U-Nets

Diffusion Transformers

References

[1] Peter Holderrieth and Ezra Erives.An Introduction to Flow Matching and Diffusion Models[EB/OL].https://arxiv.org/abs/2506.02070,2025.