论文阅读笔记:Deep Unsupervised Learning using Nonequilibrium Thermodynamics

1、来源

论文连接1:http://ganguli-gang.stanford.edu/pdf/DeepUnsupDiffusion.pdf
论文连接2(带appendix):https://arxiv.org/pdf/1503.03585v7
代码链接:https://github.com/Sohl-Dickstein/Diffusion-Probabilistic-Models
代码的环境配置(基于theano)参考:https://blog.csdn.net/u010948546/article/details/146217516?spm=1001.2014.3001.5501

2、论文推理过程

扩散模型的流程如下图所示,可以看出 q ( x 0 , 1 , 2 ⋯ , T − 1 , T ) q(x^{0,1,2\cdots ,T-1, T}) q(x0,1,2,T1,T)为正向加噪音过程, p ( x 0 , 1 , 2 ⋯ , T − 1 , T ) p(x^{0,1,2\cdots ,T-1, T}) p(x0,1,2,T1,T)为逆向去噪音过程,具体过程参考https://blog.csdn.net/u010948546/article/details/144902864?spm=1001.2014.3001.5501。可以看出,逆向去噪的末端得到的图上还散布一些噪点。
请添加图片描述

2.1、名词解释

q ( x 0 ) q(x^0) q(x0) x 0 x^0 x0 表示数据集的图像分布,例如在使用MNIST数据集时, x 0 x^0 x0就表示MNIST数据集中的图像,而 q ( x 0 ) q(x^0) q(x0)就表示数据集MNIST中数据集的分布情况。
p ( x T ) p(x^T) p(xT) x T x^T xT表示 x 0 x^0 x0的加噪结果, x T x^T xT是逆向去噪的起点,因此 p ( x T ) p(x^T) p(xT)是去噪起点的分布情况。与 π ( x T ) \pi(x^T) π(xT)相同。
值得注意的是 p ( x t ) p(x^t) p(xt) q ( x t ) q(x^t) q(xt)是相同的。

2.2、推理过程

正向加噪过程满足马尔可夫性质,因此有公式1。

q ( x 0 , 1 , 2 ⋯ , T − 1 , T ) = q ( x 0 ) ⋅ ∏ t = 1 T q ( x t ∣ x t − 1 ) = q ( x 0 ) ⋅ q ( x 1 ∣ x 0 ) ⋅ q ( x 2 ∣ x 1 ) … q ( x T ∣ x T − 1 ) . q ( x 1 , 2 ⋯ T ∣ x 0 ) = q ( x 1 ∣ x 0 ) ⋅ q ( x 2 ∣ x 1 ) … q ( x T ∣ x T − 1 ) ) . \begin{equation} \begin{split} q(x^{0,1,2\cdots,T-1,T})&=q(x^0)\cdot \prod_{t=1}^{T}{q(x^t|x^{t-1})}=q(x^0)\cdot q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1}). \\ q(x^{1,2 \cdots T}|x^0)&=q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1})). \end{split} \end{equation} q(x0,1,2,T1,T)q(x1,2Tx0)=q(x0)t=1Tq(xtxt1)=q(x0)q(x1x0)q(x2x1)q(xTxT1).=q(x1x0)q(x2x1)q(xTxT1)).

逆向去噪过程如公式2。

p θ ( x 0 , 1 , 2 ⋯ , T − 1 , T ) = p θ ( x T ) ⋅ ∏ t = 1 T p θ ( x t − 1 ∣ x t ) = p θ ( x T ) ⋅ p θ ( x T − 1 ∣ x T ) ⋅ p θ ( x T − 2 ∣ x T − 1 ) … p θ ( x 0 ∣ x 1 ) . \begin{equation} p_{\theta}(x^{0,1,2\cdots,T-1,T})=p_{\theta}(x^T)\cdot \prod_{t=1}^{T}{p_{\theta}(x^{t-1}|x^{t})}=p_{\theta}(x^T)\cdot p_{\theta}(x^{T-1}|x^T)\cdot p_{\theta}(x^{T-2}|x^{T-1})\dots p_{\theta}(x^{0}|x^{1}). \end{equation} pθ(x0,1,2,T1,T)=pθ(xT)t=1Tpθ(xt1xt)=pθ(xT)pθ(xT1xT)pθ(xT2xT1)pθ(x0x1).
公式2中的参数 θ \theta θ就是深度学习模型中需要学习的参数。为了方便,省略公式2中的 θ \theta θ,因此公式2被重写为公式3。
p ( x 0 , 1 , 2 ⋯ , T − 1 , T ) = p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) = p ( x T ) ⋅ p ( x T − 1 ∣ x T ) ⋅ p ( x T − 2 ∣ x T − 1 ) … p ( x 0 ∣ x 1 ) . \begin{equation} p(x^{0,1,2\cdots,T-1,T})=p(x^T)\cdot \prod_{t=1}^{T}{p(x^{t-1}|x^{t})}=p(x^T)\cdot p(x^{T-1}|x^T)\cdot p(x^{T-2}|x^{T-1})\dots p(x^{0}|x^{1}). \end{equation} p(x0,1,2,T1,T)=p(xT)t=1Tp(xt1xt)=p(xT)p(xT1xT)p(xT2xT1)p(x0x1).

逆向去噪的目标是使得其终点与正向加噪的起点相同。也就是使得 p ( x 0 ) p(x^0) p(x0)最大,即使得 逆向去噪过程为 x 0 x^0 x0的概率最大。

p ( x 0 ) = ∫ p ( x 0 , x 1 ) d x 1 ( 联合分布概率公式 ) = ∫ p ( x 1 ) ⋅ p ( x 0 ∣ x 1 ) d x 1 ( 贝叶斯概率公式 ) = ∫ ∫ p ( x 1 , x 2 ) d x 2 ⋅ p ( x 0 ∣ x 1 ) d x 1 ( 积分套积分 ) = ∫ ∫ p ( x 2 ) ⋅ p ( x 1 ∣ x 2 ) ⋅ p ( x 0 ∣ x 1 ) d x 1 d x 2 ( 改写为二重积分 ) = ∫ ∫ p ( x 2 ) ⋅ p ( x 1 ∣ x 2 ) ⋅ p ( x 0 ∣ x 1 ) d x 1 d x 2 = ⋮ = ∫ ∫ ⋯ ∫ p ( x T ) ⋅ p ( x T − 1 ∣ x T ) ⋅ p ( x T − 2 ∣ x − 1 ) ⋯ p ( x 0 ∣ x 1 ) ⋅ d x 1 d x 2 ⋯ d x T = ∫ p ( x 0 , 1 , 2 ⋯ T ) d x 1 , 2 ⋯ T ( T − 1 重积分 ) = ∫ d x 1 , 2 ⋯ T ⋅ p ( x 0 , 1 , 2 ⋯ T ) ⋅ q ( x 1 , 2 ⋯ T ∣ x 0 ) q ( x 1 , 2 ⋯ T ∣ x 0 ) = ∫ d x 1 , 2 ⋯ T ⋅ q ( x 1 , 2 ⋯ T ∣ x 0 ) ⋅ p ( x 0 , 1 , 2 ⋯ T ) q ( x 1 , 2 ⋯ T ∣ x 0 ) = ∫ d x 1 , 2 ⋯ T ⋅ q ( x 1 , 2 ⋯ T ∣ x 0 ) ⋅ p ( x T ) ⋅ p ( x T − 1 ∣ x T ) ⋅ p ( x T − 2 ∣ x T − 1 ) … p ( x 0 ∣ x 1 ) q ( x 1 ∣ x 0 ) ⋅ q ( x 2 ∣ x 1 ) … q ( x T ∣ x T − 1 ) = ∫ d x 1 , 2 ⋯ T ⋅ q ( x 1 , 2 ⋯ T ∣ x 0 ) ⋅ p ( x T ) ⋅ p ( x T − 1 ∣ x T ) ⋅ p ( x T − 2 ∣ x T − 1 ) … p ( x 0 ∣ x 1 ) q ( x 1 ∣ x 0 ) ⋅ q ( x 2 ∣ x 1 ) … q ( x T ∣ x T − 1 ) = ∫ d x 1 , 2 ⋯ T ⋅ q ( x 1 , 2 ⋯ T ∣ x 0 ) ⋅ p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) = E x 1 , 2 , ⋯ T ∼ q ( x 1 , 2 ⋯ T ∣ x 0 ) p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ( 改写为期望的形式 ) \begin{equation} \begin{split} p(x^0)&=\int p(x^0,x^1)dx^{1} (联合分布概率公式)\\ &=\int p(x^1)\cdot p(x^0|x^1)dx^1 (贝叶斯概率公式) \\ &=\int \int p(x1,x2)dx^2 \cdot p(x^0|x^1)dx^1 (积分套积分)\\ &=\int \int p(x^2)\cdot p(x^1|x^2) \cdot p(x^0|x^1)dx^1 dx^2(改写为二重积分)\\ &= \int \int p(x^2) \cdot p(x^1|x^2) \cdot p(x^0|x^1) dx^1 dx^2 \\ &= \vdots \\ &= \int \int \cdots \int p(x^T)\cdot p(x^{T-1}|x^{T})\cdot p(x^{T-2}|x^{-1})\cdots p(x^0|x^1) \cdot dx^1 dx^2 \cdots dx^T \\ &= \int p(x^{0,1,2 \cdots T})dx^{1,2\cdots T} (T-1重积分) \\ &= \int dx^{1,2\cdots T} \cdot p(x^{0,1,2 \cdots T}) \cdot \frac{q(x^{1,2 \cdots T}| x^0)}{q(x^{1,2 \cdots T}|x^0)} \\ &= \int dx^{1,2\cdots T} \cdot q(x^{1,2 \cdots T}| x^0) \cdot \frac{ p(x^{0,1,2 \cdots T}) }{q(x^{1,2 \cdots T}|x^0)} \\ &= \int dx^{1,2\cdots T} \cdot q(x^{1,2 \cdots T}| x^0) \cdot \frac{ p(x^T)\cdot p(x^{T-1}|x^T)\cdot p(x^{T-2}|x^{T-1})\dots p(x^{0}|x^{1})}{q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1})} \\ &= \int dx^{1,2\cdots T} \cdot q(x^{1,2 \cdots T}| x^0) \cdot p(x^T)\cdot \frac{ p(x^{T-1}|x^T)\cdot p(x^{T-2}|x^{T-1})\dots p(x^{0}|x^{1})}{q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1})} \\ &= \int dx^{1,2\cdots T} \cdot q(x^{1,2 \cdots T}| x^0) \cdot p(x^T)\cdot \prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})} \\ &= E_{x^{1,2, \cdots T} \sim q(x^{1,2 \cdots T} | x^0)} p(x^T)\cdot \prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})} (改写为期望的形式)\\ \end{split} \end{equation} p(x0)=p(x0,x1)dx1(联合分布概率公式)=p(x1)p(x0x1)dx1(贝叶斯概率公式)=∫∫p(x1,x2)dx2p(x0x1)dx1(积分套积分)=∫∫p(x2)p(x1x2)p(x0x1)dx1dx2(改写为二重积分)=∫∫p(x2)p(x1x2)p(x0x1)dx1dx2==∫∫p(xT)p(xT1xT)p(xT2x1)p(x0x1)dx1dx2dxT=p(x0,1,2T)dx1,2T(T1重积分)=dx1,2Tp(x0,1,2T)q(x1,2Tx0)q(x1,2Tx0)=dx1,2Tq(x1,2Tx0)q(x1,2Tx0)p(x0,1,2T)=dx1,2Tq(x1,2Tx0)q(x1x0)q(x2x1)q(xTxT1)p(xT)p(xT1xT)p(xT2xT1)p(x0x1)=dx1,2Tq(x1,2Tx0)p(xT)q(x1x0)q(x2x1)q(xTxT1)p(xT1xT)p(xT2xT1)p(x0x1)=dx1,2Tq(x1,2Tx0)p(xT)t=1Tq(xtxt1)p(xt1xt)=Ex1,2,Tq(x1,2Tx0)p(xT)t=1Tq(xtxt1)p(xt1xt)(改写为期望的形式)
因此公式3中的参数 θ \theta θ应满足
θ = a r g max θ p ( x 0 ) . \begin{equation} \theta= arg \underset {\theta}{\text{max}} p(x^0). \end{equation} θ=argθmaxp(x0).
公式4是对数据集中的一张图片进行求解,然而数据集中通常是有成千上万张图像的。假设数据集中有 N N N张图像,因此有公式6,其目的是求得一组参数 θ \theta θ,使得 L L L取得最大值。值得注意的是 q ( x 0 ) q(x^0) q(x0)表示数据集中每张图片被采样出来的概率。
L = ∑ n = 0 N q ( x 0 ) ⋅ l o g ( p ( x 0 ) ) = ∫ d x 0 ⋅ q ( x 0 ) ⋅ l o g ( p ( x 0 ) ) = ∫ d x 0 ⋅ q ( x 0 ) ⋅ l o g [ E x 1 , 2 , ⋯ T ∼ q ( x 1 , 2 ⋯ T ∣ x 0 ) p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] ≥ ∫ d x 0 ⋅ q ( x 0 ) ⋅ E x 1 , 2 , ⋯ T ∼ q ( x 1 , 2 ⋯ T ∣ x 0 ) l o g [ p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] = ∫ d x 0 ⋅ q ( x 0 ) ∫ q ( x 1 , 2 ⋯ T ∣ x 0 ) ⋅ l o g [ p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] ⋅ d x 1 , 2 ⋯ T = ∫ d x 0 , 1 , 2 ⋯ T q ( x 0 ) ⋅ q ( x 1 , 2 ⋯ T ∣ x 0 ) ⋅ l o g [ p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] = ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x T ) ⋅ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] = ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] + ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x T ) ] = K \begin{equation} \begin{split} L&=\sum_{n=0}^{N} q(x^0)\cdot log(p(x^0)) \\ &=\int dx^0\cdot q(x^0)\cdot log(p(x^0)) \\ &=\int dx^0\cdot q(x^0)\cdot log [ E_{x^{1,2, \cdots T} \sim q(x^{1,2 \cdots T} | x^0)} p(x^T)\cdot \prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}] \\ & \geq \int dx^0\cdot q(x^0)\cdot E_{x^{1,2, \cdots T} \sim q(x^{1,2 \cdots T} | x^0)} log [p(x^T)\cdot \prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]\\ &= \int dx^0\cdot q(x^0) \int q(x^{1,2 \cdots T}| x^0) \cdot log [p(x^T)\cdot \prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}] \cdot dx^{1,2\cdots T}\\ &= \int dx^{0,1,2\cdots T} q(x^0) \cdot q(x^{1,2 \cdots T}| x^0) \cdot log [p(x^T)\cdot \prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}] \\ &= \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log [p(x^T)\cdot \prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}] \\ &= \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log [\prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}] + \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log [p(x^T)] \\ &= K \\ \end{split} \end{equation} L=n=0Nq(x0)log(p(x0))=dx0q(x0)log(p(x0))=dx0q(x0)log[Ex1,2,Tq(x1,2Tx0)p(xT)t=1Tq(xtxt1)p(xt1xt)]dx0q(x0)Ex1,2,Tq(x1,2Tx0)log[p(xT)t=1Tq(xtxt1)p(xt1xt)]=dx0q(x0)q(x1,2Tx0)log[p(xT)t=1Tq(xtxt1)p(xt1xt)]dx1,2T=dx0,1,2Tq(x0)q(x1,2Tx0)log[p(xT)t=1Tq(xtxt1)p(xt1xt)]=dx0,1,2Tq(x0,1,2T)log[p(xT)t=1Tq(xtxt1)p(xt1xt)]=dx0,1,2Tq(x0,1,2T)log[t=1Tq(xtxt1)p(xt1xt)]+dx0,1,2Tq(x0,1,2T)log[p(xT)]=K

因此有公式
K = ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] ⏟ K 1 + ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x T ) ] ⏟ K 2 = K 1 + K 2 \begin{equation} \begin{split} K &= \underbrace{\int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log [\prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]}_{K1} + \underbrace{\int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log [p(x^T)]}_{K_2} \\ &=K_1 + K_2 \end{split} \end{equation} K=K1 dx0,1,2Tq(x0,1,2T)log[t=1Tq(xtxt1)p(xt1xt)]+K2 dx0,1,2Tq(x0,1,2T)log[p(xT)]=K1+K2

首先考虑 K K K中的第二项 K 2 K_2 K2
K 2 = ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x T ) ] = ∫ q ( x 0 ) ⋅ q ( x 1 ∣ x 0 ) ⋅ q ( x 2 ∣ x 1 ) ⋯ q ( x T ∣ x T − 1 ) ⋅ l o g [ p ( x T ) ] ⋅ d x 0 d x 1 ⋯ d x T = ∫ ( ∫ q ( x 1 , x 0 ) ⋅ d x 0 ) ⋅ q ( x 2 ∣ x 1 ) ⋯ q ( x T ∣ x T − 1 ) ⋅ l o g [ p ( x T ) ] ⋅ d x 1 ⋯ d x T = ∫ q ( x 1 ) ⋅ q ( x 2 ∣ x 1 ) ⋯ q ( x T ∣ x T − 1 ) ⋅ l o g [ p ( x T ) ] ⋅ d x 1 ⋯ d x T = ∫ q ( x T ) ⋅ l o g [ p ( x T ) ] ⋅ d x T = ∫ p ( x T ) ⋅ l o g [ p ( x T ) ] ⋅ d x T = − H p ( x T ) \begin{equation} \begin{split} K_2 &= \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log [p(x^T)] \\ &= \int q(x^0)\cdot q(x^1|x^0) \cdot q(x^2|x^1) \cdots q(x^{T}|x^{T-1})\cdot log [p(x^T)] \cdot dx^0 dx^1\cdots dx^{T} \\ &= \int \bigg( \int q(x^1, x^0) \cdot dx^0 \bigg) \cdot q(x^2|x^1) \cdots q(x^{T}|x^{T-1})\cdot log [p(x^T)] \cdot dx^1\cdots dx^{T} \\ &= \int q(x^1) \cdot q(x^2|x^1) \cdots q(x^{T}|x^{T-1})\cdot log [p(x^T)] \cdot dx^1\cdots dx^{T} \\ &= \int q(x^T) \cdot log [p(x^T)] \cdot dx^{T} \\ &= \int p(x^T) \cdot log [p(x^T)] \cdot dx^{T} \\ &=-H_p(x^T) \end{split} \end{equation} K2=dx0,1,2Tq(x0,1,2T)log[p(xT)]=q(x0)q(x1x0)q(x2x1)q(xTxT1)log[p(xT)]dx0dx1dxT=(q(x1,x0)dx0)q(x2x1)q(xTxT1)log[p(xT)]dx1dxT=q(x1)q(x2x1)q(xTxT1)log[p(xT)]dx1dxT=q(xT)log[p(xT)]dxT=p(xT)log[p(xT)]dxT=Hp(xT)
p ( x T ) p(x^T) p(xT)是一个均值为0,方差为1的高斯分布。参考【正态分布系列】正态分布的熵,可以计算出 K 2 K_2 K2如下所示。
K 2 = − H p ( x T ) = − ( 1 2 l o g [ 2 π σ 2 ] + 1 2 ) = − ( 1 2 l o g [ 2 π ] + 1 2 ) \begin{equation} \begin{split} K_2 &=-H_p(x^T) \\ &=-\bigg( \frac{1}{2} log[2 \pi \sigma^2] + \frac{1}{2} \bigg)\\ &=-\bigg( \frac{1}{2} log[2 \pi ] + \frac{1}{2} \bigg) \end{split} \end{equation} K2=Hp(xT)=(21log[2πσ2]+21)=(21log[2π]+21)
在代码中的计算过程如下图红框所示。
在这里插入图片描述
接下来考虑 K 1 K_1 K1。值得注意的是,论文中说明,为了避免边界效应,因此强迫 p ( x 0 ∣ x 1 ) = q ( x 1 ∣ x 0 ) p(x^{0}|x^1)=q(x^1|x^{0}) p(x0x1)=q(x1x0)

K 1 = ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ ∏ t = 1 T p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] = ∑ t = 1 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] + ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x 0 ∣ x 1 ) q ( x 1 ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t ∣ x t − 1 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t ) ⋅ q ( x t − 1 ) q ( x t ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ⋅ q ( x t − 1 ∣ x 0 ) q ( x t ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ] + ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ q ( x t − 1 ∣ x 0 ) q ( x t ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ] + ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ ∏ t = 2 T q ( x t − 1 ∣ x 0 ) q ( x t ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ] + ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ q ( x 1 ∣ x 0 ) q ( x T ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ] + ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ q ( x 1 ∣ x 0 ) ] − ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ q ( x T ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ] + ∫ d x 0 d x 1 ⋯ d x T ⋅ q ( x 0 ) ⋅ q ( x 1 ∣ x 0 ) ⋅ q ( x 2 ∣ x 1 ) ⋯ q ( x T ∣ x T − 1 ) ⋅ l o g [ q ( x 1 ∣ x 0 ) ] − ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ q ( x T ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ] + ∫ d x 0 d x 1 ⋅ q ( x 0 ) ⋅ q ( x 1 ∣ x 0 ) ( ∫ q ( x 2 ∣ x 1 ) ⋅ d x 2 ⏟ = 1 ) ⋅ ( ∫ q ( x 3 ∣ x 2 ) ⋅ d x 3 ⏟ = 1 ) ⋯ ( ∫ q ( x T ∣ x T − 1 ) ⋅ d x T ⏟ = 1 ) ⋅ l o g [ q ( x 1 ∣ x 0 ) ] − ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ q ( x T ∣ x 0 ) ] = ∑ t = 2 T ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ p ( x t − 1 ∣ x t ) q ( x t − 1 ∣ x t , x 0 ) ] + ∫ d x 0 d x 1 ⋅ q ( x 0 ) ⋅ q ( x 2 ∣ x 1 ) ⋅ l o g [ q ( x 1 ∣ x 0 ) ] − ∫ d x 0 , 1 , 2 ⋯ T ⋅ q ( x 0 , 1 , 2 ⋯ T ) ⋅ l o g [ q ( x T ∣ x 0 ) ] \begin{equation} \begin{split} K_1 &=\int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\prod_{t=1}^{T} \frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\bigg] \\ &= \sum_{t=1}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\bigg] \\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\bigg] + \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{0}|x^1)}{q(x^1|x^{0})}\bigg] \\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\bigg] \\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t})}\cdot \frac{q(x^{t-1})}{q(x^t)}\bigg] \\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\cdot \frac{q(x^{t-1}|x^0)}{q(x^t|x^0)}\bigg] \\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\bigg] + \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{q(x^{t-1}|x^0)}{q(x^t|x^0)}\bigg]\\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\bigg] + \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\prod_{t=2}^{T} \frac{q(x^{t-1}|x^0)}{q(x^t|x^0)}\bigg]\\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\bigg] + \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{q(x^{1}|x^0)}{q(x^T|x^0)}\bigg]\\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\bigg] + \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[q(x^{1}|x^0)\bigg] - \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[q(x^T|x^0)\bigg]\\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\bigg] + \int dx^{0}dx^{1} \cdots dx^{T} \cdot q(x^{0}) \cdot q(x^1|x^0) \cdot q(x^2|x^1) \cdots q(x^T|x^{T-1}) \cdot log \bigg[q(x^{1}|x^0)\bigg] - \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[q(x^T|x^0)\bigg]\\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\bigg] + \int dx^{0}dx^{1} \cdot q(x^{0}) \cdot q(x^1|x^0) \bigg( \underbrace{\int q(x^2|x^1) \cdot dx^{2}}_{=1} \bigg) \cdot \bigg( \underbrace{\int q(x^3|x^2) \cdot dx^{3}}_{=1} \bigg) \cdots \bigg( \underbrace{\int q(x^T|x^{T-1}) \cdot dx^{T}}_{=1} \bigg) \cdot log \bigg[q(x^{1}|x^0)\bigg] - \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[q(x^T|x^0)\bigg]\\ &= \sum_{t=2}^{T} \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[\frac{ p(x^{t-1}|x^t)}{q(x^{t-1}|x^{t}, x^0)}\bigg] + \int dx^{0}dx^{1} \cdot q(x^{0}) \cdot q(x^2|x^1) \cdot log \bigg[q(x^{1}|x^0)\bigg] - \int dx^{0,1,2\cdots T} \cdot q(x^{0,1,2 \cdots T}) \cdot log \bigg[q(x^T|x^0)\bigg]\\ \end{split} \end{equation} K1=dx0,1,2Tq(x0,1,2T)log[t=1Tq(xtxt1)p(xt1xt)]=t=1Tdx0,1,2Tq(x0,1,2T)log[q(xtxt1)p(xt1xt)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xtxt1)p(xt1xt)]+dx0,1,2Tq(x0,1,2T)log[q(x1x0)p(x0x1)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xtxt1)p(xt1xt)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt)p(xt1xt)q(xt)q(xt1)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)q(xtx0)q(xt1x0)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)]+t=2Tdx0,1,2Tq(x0,1,2T)log[q(xtx0)q(xt1x0)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)]+dx0,1,2Tq(x0,1,2T)log[t=2Tq(xtx0)q(xt1x0)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)]+dx0,1,2Tq(x0,1,2T)log[q(xTx0)q(x1x0)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)]+dx0,1,2Tq(x0,1,2T)log[q(x1x0)]dx0,1,2Tq(x0,1,2T)log[q(xTx0)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)]+dx0dx1dxTq(x0)q(x1x0)q(x2x1)q(xTxT1)log[q(x1x0)]dx0,1,2Tq(x0,1,2T)log[q(xTx0)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)]+dx0dx1q(x0)q(x1x0)(=1 q(x2x1)dx2)(=1 q(x3x2)dx3)(=1 q(xTxT1)dxT)log[q(x1x0)]dx0,1,2Tq(x0,1,2T)log[q(xTx0)]=t=2Tdx0,1,2Tq(x0,1,2T)log[q(xt1xt,x0)p(xt1xt)]+dx0dx1q(x0)q(x2x1)log[q(x1x0)]dx0,1,2Tq(x0,1,2T)log[q(xTx0)]

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/898309.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

7种数据结构

7种数据结构 顺序表sqlite.hseqlite.c 单链表linklist.clinklist.h 双链表doulinklist.cdoulinklist.h 链式栈linkstack.clinkstack.h 队列SeqQueue.cSeqQueue.h 树tree.c 哈希表hash.c 顺序表 sqlite.h #ifndef __SEQLIST_H__ #define __SEQLIST_H__ typedef struct person…

Linux 查看及测试网络命令

使用 ifconfig 命令查看网络接口地址 查看指定的网络接口信息 执行 ifconfig ens33 命令可以只查看网卡 ens33 的配置信息

ABAP语言的动态编程(4) - 综合案例:管理费用明细表

本篇来实现一个综合案例:管理费用明细表。报表在实际项目中,也有一定的参考意义,一方面展示类似的报表,比如管理费用、研发费用等费用的明细,使用业务比较习惯的展示格式;另一方面正好综合运用前面学习的动…

【Redis】Redis的数据删除(过期)策略,数据淘汰策略。

如果问到:假如Redis的key过期之后,会立即删除吗? 其实就是想问数据删除(过期)策略。 如果面试官问到:如果缓存过多,内存是有限的,内存被占满了怎么办? 其实就是问:数据的淘汰策略。…

Linux配置yum仓库,服务控制,防火墙

一、yum仓库 1.在安装软件时,首先第一步就是要考虑软件的版本的问题! 2.软件的安装:最安全可靠的方法就是去软件对应的官网上查看安装手册(包括的软件的下载) 红帽系软件安装的常见的3种方式 (1&#x…

[从零开始学习JAVA] Stream流

前言: 本文我们将学习Stream流,他就像流水线一样,可以对我们要处理的对象进行逐步处理,最终达到我们想要的效果,是JAVA中的一大好帮手,值得我们了解和掌握。(通常和lambda 匿名内部类 方法引用相…

设计模式(创建型)-抽象工厂模式

摘要 在软件开发的复杂世界中,设计模式作为解决常见问题的最佳实践方案,一直扮演着至关重要的角色。抽象工厂模式,作为一种强大的创建型设计模式,在处理创建一系列或相关依赖对象的场景时,展现出了独特的优势和灵活性。它通过提供一个创建对象的接口,让开发者能够在不指定…

【JavaEE】-- SpringBoot快速上手

文章目录 1. Maven1.1 什么是Maven1.2 为什么要学Maven1.3 创建一个Maven项目1.4 Maven核心功能1.4.1 项目创建1.4.2 依赖管理1.4.3 Maven Help插件 1.5 Maven仓库1.5.1 本地仓库1.5.2 中央仓库1.5.3 私有服务器(私服) 1.6 Maven设置国内源1.6.1 配置当前…

TCP/IP 协议精讲-精华总结版本

序言 本文旨在介绍一下TCP/IP涉及得所有基础知识,为大家从宏观上俯瞰TCP/IP提供一个基石,文档属于《TCP/IP图解(第五版)》的精简版本。 专业术语 缩写 全称 WAN Wide area network广域网 LAN Local area network局域网 TC…

Leetcode 160 Intersection of Two Linked Lists

题意 给定两个链表,找这两个链表第一个公共节点,如果没有返回nullptr 题目链接 https://leetcode.com/problems/intersection-of-two-linked-lists/description/ 题解 两个指针分别从两个链表(记录为表A,表B)的表…

C语言——结构体、联合、枚举

C语言中自定义类型 结构体结构体变量的创建和初始化结构体传参结构体内存对齐(如何存储) 联合体(共用体)联合体创建和初始化联合体大小(如何存储) 枚举类型枚举类型创建枚举类型初始化枚举的优点(相较于define) 前言 C语言中有内置类型和自定义类型,内置类型就像int…

利用pprof对golang进行性能分析

利用pprof进行性能分析 pprof性能分析的5个方面 一、性能分析的五个核心维度 CPU分析 - 剖析程序的CPU使用情况,定位高耗时函数 内存分析 - 追踪内存分配与泄露,优化内存使用模式 IO分析 - 监控文件/网络IO操作,发现瓶颈资源 Goroutine分…

IntelliJ IDEA 调试技巧指南

在日常开发中,调试是不可或缺的一部分。掌握调试工具的使用可以让我们更高效地定位和解决问题。本文将介绍一些在 IntelliJ IDEA 中常用的调试技巧,希望能帮助你在开发过程中更顺畅地解决问题。 1. 方法断点:快速定位实现类 方法断点可以帮…

gitlab 提交pr

在 GitLab 中,提交合并请求(Merge Request, MR)的大致流程如下: 1. 创建新分支 如果你还没有创建新的功能分支,可以使用以下命令创建并切换到新分支: git checkout -b feature-branch说明:f…

halcon几何测量(二)计算距离和角度的函数

目录 一、计算两条线之间的夹角二、计算一条直线和水平轴之间的夹角三、计算两个轮廓之间的最小距离四、计算两个轮廓之间的最小距离和对应的点五、计算直线和区域之间的最小和最大距离六、计算点到轮廓线之间的距离七、计算点到直线的距离八、计算点到点的距离九、计算点和区域…

【Linux操作系统——学习笔记二】Linux简单导航命令操作

一、前言 学习Linux,本质上是学习在命令行下熟练使用Linux的各类命令。 命令行:是一种通过输入命令和参数与计算机系统进行交互的方式,可以使用各种字符化命令对系统发出操作指令,打开Linux终端,进入命令行界面。 …

新安装的cursor安装不了插件

我安装的cursor版本0.47.5 直接说解决办法 找到安装路径cursor\resources\app下的product.json 修改https://marketplace.cursorapi.com为https://marketplace.visualstudio.com

算法基础篇(蓝桥杯常考点)

算法基础篇 前言 算法内容还有搜索,数据结构(进阶),动态规划和图论 数学那个的话大家也知道比较难,放在最后讲 这期包含的内容可以看目录 模拟那个算法的话就是题说什么写什么,就不再分入目录中了 注意事…

《解锁华为黑科技:MindSpore+鸿蒙深度集成奥秘》

在数字化浪潮汹涌澎湃的当下,人工智能与操作系统的融合已成为推动科技发展的核心驱动力。华为作为科技领域的先锋,其AI开发框架MindSpore与鸿蒙系统的深度集成备受瞩目,开启了智能生态的新篇章。 华为MindSpore:AI框架的创新先锋…

双3060、Ubuntu22.04、cuda12.8安装deepseek 32b-Q8

以下是针对双RTX 3060显卡(12GB显存)在Ubuntu 22.04系统部署DeepSeek-R1-32b-qwen-distill-q8模型的完整流程,结合最新技术规范与魔塔社区资源: 一、驱动与CUDA环境配置 1. 禁用开源驱动 bash sudo tee /etc/modprobe.d/blackli…