Faster Unbalanced Optimal Transport: Translation invariant Sinkhorn and 1-D Frank-Wolfe阅读笔记

$m(\alpha )\triangleq ⟨\alpha ,1⟩={\sum }_i{\alpha }_i$

OT

概率向量$(\alpha ,\beta )\in {\mathbb{R}}_{+}^N\times {\mathbb{R}}_{+}^M$， 满足${\sum }_i{\alpha }_i={\sum }_j{\beta }_j=1$

代价矩阵$\mathrm{C}\in {\mathbb{R}}^{N\times M}$
$$\mathrm{OT}(\alpha ,\beta )\triangleq \underset{\pi ⩾0,{\pi }_1=\alpha ,{\pi }_2=\beta }{\inf }⟨\pi ,\mathrm{C}⟩=\sum _{i,j}{\pi }_{i,j}{\mathrm{C}}_{i,j}$$
其中$\left({\pi }_1,{\pi }_2\right)\triangleq \left(\pi 1,{\pi }^{\top }1\right)$

Csiszár divergences

entropy function$\phi :{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$

满足大于等于0，凸函数，下半连续，$\varphi \left(1\right)=0$

定义${\phi }_{\infty }^{\prime }\triangleq {\lim }_{x\to \infty }\frac{\phi (x)}{x}$

那么Csiszár divergences
$${\mathrm{D}}_{\phi }(\mu \mid \nu )\triangleq \sum _{{\nu }_i>0}\phi \left(\frac{{\mu }_i}{{\nu }_i}\right){\nu }_i+{\phi }_{\infty }^{\prime }\sum _{{\nu }_i=0}{\mu }_i$$
一种实例就是KL散度

$\phi (x)=x\log x-x+1$
$$\mathrm{KL}(\mu \mid \nu )\triangleq \sum _i\left[\log \left(\frac{{\mu }_i}{{\nu }_i}\right){\mu }_i-{\mu }_i+{\nu }_i\right]$$

Legendre transform

设区$I\subset \mathbb{R}$,$f:I\to \mathbb{R}$是一个凸函数，则$f$的Legendre transform为$f^{\ast }:I^{\ast }\to \mathbb{R}$
$$f^{\ast }\left(x^{\ast }\right)=\underset{x\in I}{\sup }\left(x^{\ast }x-f\left(x\right)\right),x^{\ast }\in I^{\ast }$$
其中 I ∗ = { x ∗ ∈ R : f ∗ ( x ∗ ) < ∞ } I^* = \left\{x^*\in \mathbb{R}:f^*\left(x^*\right)<\infty\right\} I∗={ x∗∈R:f∗(x∗)<∞}

类似地

定义在凸集$\mathbf{x}\subset {\mathbb{R}}^n$的凸函数$f:X\to \mathbb{R}$,则$f^{\ast }:X^{\ast }\to \mathbb{R}$
$$f^{\ast }\left({\mathbf{x}}^{\ast }\right)=\underset{\mathbf{x}\in X}{\sup }\left(⟨{\mathbf{x}}^{\ast },\mathbf{x}⟩-f\left(x\right)\right),{\mathbf{x}}^{\ast }\in X^{\ast }$$
其中 X ∗ = { x ∗ ∈ R n : sup ⁡ x ∈ X ( ⟨ x ∗ , x ⟩ − f ( x ) ) < ∞ } X^*=\left\{\mathbf{x}^* \in \mathbb{R}^n: \sup_{\mathbf{x} \in X}\left(\left\langle \mathbf{x}^*, \mathbf{x}\right\rangle-f(\mathbf{x})\right)<\infty\right\} X∗={ x∗∈Rn:supx∈X​(⟨x∗,x⟩−f(x))<∞}

性质

Separable sum

$$f\left(x_1,x_2\right)=g\left(x_1\right)+h\left(x_2\right)f^{\ast }\left(y_1,y_2\right)=g^{\ast }\left(y_1\right)+h^{\ast }\left(y_2\right)$$

放缩

$$\begin{array}{cc}f(x)=\alpha g(x)& f^{\ast }(y)=\alpha g^{\ast }(y/\alpha )\\ f(x)=\alpha g(x/\alpha )& f^{\ast }(y)=\alpha g^{\ast }(y)\end{array}$$

平移

$$f(x)=g(x-b)f^{\ast }(y)=b^{T}y+g^{\ast }(y)$$

加上仿射函数

$$f(x)=g(x)+a^{T}x+b{f}^{\ast }(y)=g^{\ast }(y-a)-b$$

可逆仿射变换

设$A$是非奇异方阵
$$f(x)=g(Ax)f^{\ast }(y)=g^{\ast }\left(A^{-T}y\right)$$

实例

指示函数

ι { 1 } ( x ) = { 0 , x = 1 + ∞ , otherwise \iota_{\{1\}}(x)=\begin{cases}0, &x=1\\+\infty, &\text{otherwise}\\\end{cases} ι{ 1}​(x)={ 0,+∞,​x=1otherwise​
ι { 1 } ∗ ( y ) = y \iota_{\{1\}}^*(y)=y ι{ 1}∗​(y)=y

KL散度

$f\left(\mu \right)=\mathrm{KL}(\mu \mid \nu )={\sum }_i\left[\log \left(\frac{{\mu }_i}{{\nu }_i}\right){\mu }_i-{\mu }_i+{\nu }_i\right]$

证明：
$$g\left(\mathbf{y}\right)={\mathbf{y}}^T\mu -\sum _i\left[\log \left(\frac{{\mu }_i}{{\nu }_i}\right){\mu }_i-{\mu }_i+{\nu }_i\right]$$

$$\frac{\partial g}{\partial {\mu }_i}=y_i-\log \frac{{\mu }_i}{{\nu }_i}=0\Rightarrow {\mu }_i=v_i{e}^{y_i}$$

因此
$$f^{\ast }\left(\mathbf{y}\right)=\sum _i\left(v_i{e}^{y_i}-v_i\right)=⟨\mathbf{v},e^{\mathbf{y}}-1⟩$$

例子3

$\phi (x)=x\log x-x+1$
$${\phi }^{\ast }\left(y\right)=e^y-1$$

Unbalanced optimal transport

$$\begin{array}{rl}& \mathrm{UOT}(\alpha ,\beta )\triangleq \underset{\pi ⩾0}{\inf }⟨\pi ,\mathrm{C}⟩+\epsilon \mathrm{KL}(\pi \mid \alpha \otimes \beta )\\ & +{\mathrm{D}}_{{\phi }_1}\left({\pi }_1\mid \alpha \right)+{\mathrm{D}}_{{\phi }_2}\left({\pi }_2\mid \beta \right)\end{array}$$

其中$m\left(\alpha \right)$不一定等于$m\left(\beta \right)$

对偶

$$\mathrm{UOT}(\alpha ,\beta )=\underset{(f,g)}{\sup }{\mathcal{F}}_{\epsilon }(f,g)$$

其中
$$\begin{array}{rl}{\mathcal{F}}_{\epsilon }(f,g)\triangleq & ⟨\alpha ,-{\phi }_1^{\ast }(-f)⟩+⟨\beta ,-{\phi }_2^{\ast }(-g)⟩\\ & -\epsilon ⟨\alpha \otimes \beta ,e^{\frac{f\oplus g-\mathrm{C}}{\epsilon }}-1⟩\end{array}$$
其中${\phi }^{\ast }$是Legendre transform

${\phi }_1^{\ast }(-f)\triangleq {\left({\phi }_1^{\ast }\left(-f_i\right)\right)}_i\in {\mathbb{R}}^N$

如果$ϵ=0$, 则最后一项变为约束$f\oplus g\le C$

Sinkhorn算法

一种解决OT/UOT的对偶问题方法就是Sinkhorn算法

当$D_{\phi }=\rho KL$时，收敛速度为${\left(1+\frac{ϵ}{\rho }\right)}^{-1}$,当$\epsilon \ll \rho$时，趋于$1$

平移不变公式

当 φ 1 ( x ) = φ 2 ( x ) = ι { 1 } ( x ) \varphi_1(x) = \varphi_2(x)=\iota_{\{1\}}(x) φ1​(x)=φ2​(x)=ι{ 1}​(x)时
$${\mathcal{F}}_{\epsilon }(f,g)=⟨\alpha ,f⟩+⟨\beta ,g⟩-\epsilon ⟨\alpha \otimes \beta ,e^{\frac{f\oplus g-\mathrm{C}}{\epsilon }}-1⟩$$
对于任意常数$\lambda \in \mathbb{R},{\mathcal{F}}_{\epsilon }(f+\lambda ,g-\lambda )={\mathcal{F}}_{\epsilon }(f,g)+\lambda m\left(\alpha \right)-\lambda m\left(\beta \right)={\mathcal{F}}_{\epsilon }(f,g)$

但是对于一般的UOT不成立

设$\left(f^{\ast },g^{\ast }\right)$是${\mathcal{F}}_{\epsilon }$的最优解。如果Sinkhorn算法的初值为$f_0=f^{\star }+\tau ,\tau \in \mathbb{R}$

对于$D_{\phi }=\rho KL$,有$f_t=f^{\ast }+{\left(\frac{\rho }{\epsilon +\rho }\right)}^{2t}\tau$,即迭代对平移敏感，并且当$\epsilon \ll \rho$时，误差减小的速度很慢

为了解决这个问题，作者提出
$${\mathcal{H}}_{\epsilon }(\bar{f},\bar{g})\triangleq \underset{\lambda \in \mathbb{R}}{\sup }{\mathcal{F}}_{\epsilon }(\bar{f}+\lambda ,\bar{g}-\lambda )$$
此时${\mathcal{H}}_{\epsilon }(\bar{f}+\lambda ,\bar{g}-\lambda )={\mathcal{H}}_{\epsilon }(\bar{f},\bar{g})$

对于$UOT\left(\alpha ,\beta \right)$,${\mathcal{F}}_{\epsilon }(f,g)$和 ${\mathcal{H}}_{\epsilon }(\bar{f},\bar{g})$产生相同的解
$$\begin{array}{cl}& (f,g)=\left(\bar{f}+{\lambda }^{\star }(\bar{f},\bar{g}),\bar{g}-{\lambda }^{\star }(\bar{f},\bar{g})\right)\\ \text{ where }& {\lambda }^{\star }(\bar{f},\bar{g})\triangleq \underset{\lambda \in \mathbb{R}}{\mathrm{argmax}}{\mathcal{F}}_{\epsilon }(\bar{f}+\lambda ,\bar{g}-\lambda ).\end{array}$$
作者假设${\phi }^{\ast }$严格凸函数，进而${\lambda }^{\ast }$唯一

$H_{\epsilon }$性质

$m(\alpha )\triangleq ⟨\alpha ,1⟩={\sum }_i{\alpha }_i$

${\lambda }^{\star }(\bar{f},\bar{g})\triangleq \underset{\lambda \in \mathbb{R}}{\mathrm{argmax}}{\mathcal{F}}_{\epsilon }(\bar{f}+\lambda ,\bar{g}-\lambda )$

性质1

设${\phi }_1^{\ast },{\phi }_2^{\ast }$是光滑，严格凸函数

则存在一个唯一的极大值解${\lambda }^{\star }(\bar{f},\bar{g})$

进一步，$(\tilde{\alpha },\tilde{\beta })=\nabla {\mathcal{H}}_0(\bar{f},\bar{g})$满足$\tilde{\alpha }=\nabla {\phi }_1^{\ast }\left(-\bar{f}-{\lambda }^{\ast }(\bar{f},\bar{g})\right)\alpha ,\tilde{\beta }=\nabla {\phi }_2^{\ast }\left(-\bar{g}+{\lambda }^{\star }(\bar{f},\bar{g})\right)\beta$

并且$m(\tilde{\alpha })=m(\tilde{\beta })$

(这里${\phi }^{\ast }$是一个标量函数，因此这个$\nabla {\phi }^{\ast }$应该是一个对角矩阵，即$\nabla {\phi }^{\ast }\left(f\right)=\mathrm{diag}\left(\begin{array}{c}{\left({\phi }^{\ast }\right)}^{\prime }\left(f_1\right)\\ {\left({\phi }^{\ast }\right)}^{\prime }\left(f_2\right)\\ ⋮\\ {\left({\phi }^{\ast }\right)}^{\prime }\left(f_n\right)\end{array}\right)$)

证明：

对于任意的$(\bar{f},\bar{g})$，定义${\mathcal{G}}_{\epsilon }(\lambda )\stackrel{\text{ def. }}{=}{\mathcal{F}}_{\epsilon }(\bar{f}+\lambda ,\bar{g}-\lambda )$

根据[Liero et al., 2015], $\underset{x\to \infty }{\lim }\phi \left(x\right)=+\infty$, 当$\lambda \to \pm \infty$时，${\mathcal{G}}_{\epsilon }\to -\infty$,即${\mathcal{G}}_{\epsilon }$是强制函数

因此能在$\mathbb{R}$取到全局最大值

因为${\phi }^{\ast }$是严格凸的，所以唯一

对${\mathcal{G}}_{\epsilon }$求导得
$$\begin{array}{rl}& \frac{\mathrm{d}{\mathcal{G}}_{\epsilon }}{\mathrm{d}\lambda }=⟨\alpha ,\nabla {\phi }_1^{\ast }\left(-\bar{f}-\lambda \right)⟩-⟨\beta ,\nabla {\phi }_2^{\ast }(-\bar{g}+\lambda )⟩=0\\ & \Rightarrow ⟨\alpha ,\nabla {\phi }_1^{\ast }\left(-\bar{f}-\lambda \right)⟩=⟨\beta ,\nabla {\phi }_2^{\ast }(-\bar{g}+\lambda )⟩\\ & \Rightarrow ⟨\tilde{\alpha },1⟩=⟨\tilde{\beta },1⟩\\ & \Rightarrow m(\tilde{\alpha })=m(\tilde{\beta })\end{array}$$

性质2

设${\phi }_i={\rho }_{i}KL$

$${\lambda }^{\star }(\bar{f},\bar{g})=\frac{{\rho }_1{\rho }_2}{{\rho }_1+{\rho }_2}\log \left[\frac{⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩}{⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩}\right]$$
证明：
$$\begin{array}{rl}& ⟨\alpha ,e^{-\frac{\bar{f}+{\lambda }^{\star }}{{\rho }_1}}⟩=⟨\beta ,e^{-\frac{\bar{g}-{\lambda }^{\star }}{{\rho }_2}}⟩,\\ & \iff e^{-\frac{{\lambda }^{\star }}{{\rho }_1}}⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩=e^{+\frac{{\lambda }^{\star }}{{\rho }_2}}⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩,\\ & \iff -\frac{{\lambda }^{\star }}{{\rho }_1}+\log ⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩=\frac{{\lambda }^{\star }}{{\rho }_2}+\log ⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩,\\ & \iff {\lambda }^{\star }\left(\frac{1}{{\rho }_1}+\frac{1}{{\rho }_2}\right)=\log \left[\frac{⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩}{⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩}\right],\\ & \iff {\lambda }^{\star }(\bar{f},\bar{g})=\frac{{\rho }_1{\rho }_2}{{\rho }_1+{\rho }_2}\log \left[\frac{⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩}{⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩}\right].\end{array}$$

性质3

令${\tau }_1=\frac{{\rho }_1}{{\rho }_1+{\rho }_2}，{\tau }_2=\frac{{\rho }_2}{{\rho }_1+{\rho }_2}$

则

$$\begin{array}{r}{\mathcal{H}}_{\epsilon }(\bar{f},\bar{g})={\rho }_{1}m(\alpha )+{\rho }_{2}m(\beta )-\epsilon ⟨\alpha \otimes \beta ,e^{\frac{\bar{f}\oplus \bar{g}-\mathrm{C}}{\epsilon }}-1⟩\\ -\left({\rho }_1+{\rho }_2\right){\left(⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩\right)}^{{\tau }_1}{\left(⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩\right)}^{{\tau }_2}\cdot \end{array}$$
特别地，当${\rho }_1={\rho }_2=\rho ,\epsilon =0$时
$${\mathcal{H}}_0(\bar{f},\bar{g})=\rho \left[m(\alpha )+m(\beta )-2\sqrt{⟨\alpha ,e^{-\frac{\bar{f}}{\rho }}⟩⟨\beta ,e^{-\frac{\bar{g}}{\rho }}⟩}\right]$$
证明：

${\phi }_i(x)={\rho }_i\left(x\log x-x+1\right)$
$${\phi }_i^{\ast }\left(x\right)={\rho }_i\left(e^{\frac{x}{{\rho }_i}}-1\right)$$

$$\begin{array}{rl}{\mathcal{F}}_0(f,g)& =⟨\alpha ,-{\rho }_1\left(e^{-\frac{\bar{f}}{{\rho }_1}}-1\right)⟩+⟨\beta ,-{\rho }_2\left(e^{-\frac{\bar{g}}{{\rho }_2}}-1\right)⟩\\ & ={\rho }_{1}m(\alpha )+{\rho }_{2}m(\beta )-{\rho }_1⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩-{\rho }_2⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩\end{array}$$

根据性质2
$$\begin{array}{rl}⟨\alpha ,e^{-\frac{\bar{f}+{\lambda }^{\star }(\bar{f},\bar{g})}{{\rho }_1}}⟩& =⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩\cdot \exp \left(-\frac{{\rho }_2}{{\rho }_1+{\rho }_2}\log \left[\frac{⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩}{⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩}\right]\right)\\ & =⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩\cdot {⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩}^{-\frac{{\rho }_2}{{\rho }_1+{\rho }_2}}\cdot {⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩}^{\frac{{\rho }_2}{{\rho }_1+{\rho }_2}}\\ & ={⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩}^{\frac{{\rho }_1}{{\rho }_1+{\rho }_2}}\cdot {⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩}^{\frac{{\rho }_2}{{\rho }_1+{\rho }_2}}\end{array}$$
类似地
$$⟨\beta ,e^{-\frac{\bar{g}-{\lambda }^{\star }(\bar{f},\bar{g})}{{\rho }_2}}⟩={⟨\alpha ,e^{-\frac{\bar{f}}{{\rho }_1}}⟩}^{\frac{{\rho }_1}{{\rho }_1+{\rho }_2}}\cdot {⟨\beta ,e^{-\frac{\bar{g}}{{\rho }_2}}⟩}^{\frac{{\rho }_2}{{\rho }_1+{\rho }_2}}$$
全部代回去即可得到结论

Translation invariant Sinkhorn

Unbalanced Sinkhorn

对于任意初值$f_0$
$$\begin{array}{rl}& g_{t+1}(y)=-{\mathrm{aprox}}_{{\phi }_1^{\ast }}\left(-{\mathrm{Smin}}_{\epsilon }^{\alpha }\left(\mathrm{C}(\cdot ,y)-f_t\right)\right)\\ & f_{t+1}(x)=-{\mathrm{aprox}}_{{\phi }_2^{\ast }}\left(-{\mathrm{Smin}}_{\epsilon }^{\beta }\left(\mathrm{C}(x,\cdot )-g_{t+1}\right)\right)\end{array}$$
其中softmin定义为${\mathrm{Smin}}_{\epsilon }^{\alpha }(f)\triangleq -\epsilon \log ⟨\alpha ,e^{-f/\epsilon }⟩$
anisotropic prox为
$${\mathrm{aprox}}_{{\phi }^{\ast }}(x)\triangleq \arg \underset{y\in \mathbb{R}}{\min }\epsilon e^{\frac{x-y}{\epsilon }}+{\phi }^{\ast }(y)$$
如果$\phi =\rho KL$，则${\mathrm{aprox}}_{{\phi }^{\ast }}(x)=\frac{\rho }{\epsilon +\rho }x$

softmin和${\mathrm{aprox}}_{{\phi }^{\ast }}$在$\Vert \cdot {\Vert }_{\infty }$下分别是1-压缩和${\left(1+\frac{\epsilon }{\rho }\right)}^{-1}$-压缩的

TI-Sinkhorn

对${\mathcal{H}}_{\epsilon }$进行交替对偶上升
令
$$\begin{gathered} {\Psi }_1\left(\bar{g}\right)\triangleq \arg \max {\mathcal{H}}_{\epsilon }\left(\cdot ,\bar{g}\right) \\ {\Psi }_2\left(\bar{f}\right)\triangleq \arg \max {\mathcal{H}}_{\epsilon }\left(\bar{f},\cdot \right) \end{gathered}$$
TI-Sinkhorn算法为
$${\bar{g}}_{t+1}={\Psi }_2\left({\bar{f}}_t\right),{\bar{f}}_{t+1}={\Psi }_2\left({\bar{g}}_{t+1}\right)$$

最后
$$\left(f_t,g_t\right)\triangleq \left({\bar{f}}_t+{\lambda }^{\ast }\left({\bar{f}}_t,{\bar{g}}_t\right),{\bar{g}}_t-{\lambda }^{\ast }\left({\bar{f}}_t,{\bar{g}}_t\right)\right)$$

这个算法继承了${\mathcal{H}}_{\epsilon }$的不变性
${\Psi }_1\left({\bar{f}}_t+\mu \right)={\Psi }_1\left({\bar{f}}_t\right)-\mu$
即，如果${\bar{f}}_t$变为${\bar{f}}_t+\mu$,则${\bar{g}}_{t+1}$变为${\bar{g}}_{t+1}-\mu$
或者如果${\bar{f}}_t\to {\bar{f}}_t+\mu$，则${\bar{g}}_{t+1}\to {\bar{g}}_{t+1}-\mu$

性质4：对于固定的$\left(\bar{g},\bar{f}\right)$
令$\hat{f}\triangleq {\text{Smin}}_{\epsilon }^{\alpha }\left(C\left(x,\cdot \right)-\bar{f}\right)$
$\hat{g}\triangleq {\text{Smin}}_{\epsilon }^{\alpha }\left(C\left(\cdot ,y\right)-\bar{g}\right)$
${\psi }_1\stackrel{\text{ def. }}{=}{\Psi }_1(\bar{g}),{\psi }_2\stackrel{\text{ def. }}{=}{\Psi }_2(\bar{g})$

则
$$\begin{array}{rl}& {\psi }_1=-{\mathrm{aprox}}_{{\phi }_2^{\ast }}\left(-\hat{g}+{\lambda }^{\star }\left({\psi }_1,\bar{g}\right)\right)-{\lambda }^{\star }\left({\psi }_1,\bar{g}\right),\\ & {\psi }_2=-{\mathrm{aprox}}_{{\phi }_1^{\ast }}\left(-\hat{f}-{\lambda }^{\star }\left(\bar{f},{\psi }_2\right)\right)+{\lambda }^{\star }\left(\bar{f},{\psi }_2\right).\end{array}$$

证明：
$${\mathcal{H}}_{\epsilon }(\bar{f},\bar{g})={\mathcal{F}}_{\epsilon }\left(\bar{f}+{\lambda }^{\star }(\bar{f},\bar{g}),\bar{g}-{\lambda }^{\star }(\bar{f},\bar{g})\right)$$

对$\bar{g}$求导
$$\begin{array}{rl}\beta \odot e^{\bar{g}/\epsilon }⟨\alpha ,e^{(\bar{f}-\mathrm{C})/\epsilon }⟩& =\beta \odot \nabla {\phi }_2^{\ast }\left(-\bar{g}+{\lambda }^{\star }(\bar{f},\bar{g})\right)\\ e^{\bar{g}/\epsilon }⟨\alpha ,e^{(\bar{f}-\mathrm{C})/\epsilon }⟩& =\nabla {\phi }_2^{\ast }\left(-\bar{g}+{\lambda }^{\star }(\bar{f},\bar{g})\right)\end{array}$$
对$\bar{f}$求导
$$e^{\bar{f}/\epsilon }⟨\alpha ,e^{(\bar{g}-\mathrm{C})/\epsilon }⟩=\nabla {\phi }_1^{\ast }\left(-\bar{f}-{\lambda }^{\star }(\bar{f},\bar{g})\right)$$
令$\hat{g}=\bar{g}-{\lambda }^{\star }(\bar{f},\bar{g})$
$$e^{\hat{g}/\epsilon }⟨\beta ,e^{\left(\bar{f}+{\lambda }^{\star }(\bar{f},\bar{g})-\mathrm{C}\right)/\epsilon }⟩=\nabla {\phi }_2^{\ast }(-\hat{g})$$
$$\hat{g}=\bar{g}-{\lambda }^{\star }(\bar{f},\bar{g})=-{\mathrm{aprox}}_{{\phi }_2^{\ast }}\left(-{\mathrm{Smin}}_{\epsilon }^{\alpha }\left(\mathrm{C}-\bar{f}-{\lambda }^{\star }(\bar{f},\bar{g})\right)\right)$$

由$\bar{g}={\Psi }_1(\bar{f})$，因此结论成立

剩下的
还在看= =