Cov2d反向传播梯度的计算过程

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/diyoosjtu/article/details/89843359

我们用一个例子来说明：

令
$x*w=y$
并且
$x=\begin{bmatrix} x_{11} & x_{12} & x_{13}\\ x_{21} & x_{22} & x_{23}\\ x_{31} & x_{32} & x_{33} \end{bmatrix}, \quad w=\begin{bmatrix} w_{11} & w_{12}\\ w_{21} & w_{22} \end{bmatrix}, \quad y=\begin{bmatrix} y_{11} & y_{12}\\ y_{21} & y_{22} \end{bmatrix}$

则
$\left\{\begin{aligned} &y_{11} = w_{11}x_{11} + w_{12}x_{12}+w_{21}x_{21}+w_{22}x_{22}\\ &y_{12} = w_{11}x_{12} + w_{12}x_{13}+w_{21}x_{22}+w_{22}x_{23}\\ &y_{21} = w_{11}x_{21} + w_{12}x_{22}+w_{21}x_{31}+w_{22}x_{32}\\ &y_{22} = w_{11}x_{22} + w_{12}x_{23}+w_{21}x_{32}+w_{22}x_{33} \end{aligned}\right.$

梯度
$\frac{\partial L}{\partial x} = \frac{\partial L}{\partial y}\frac{\partial y}{\partial x}=\begin{bmatrix} \delta_{11}w_{11} & \delta_{11}w_{12} + \delta_{12}w_{11} & \delta_{12}w_{12}\\ \delta_{11}w_{21} + \delta_{21}w_{11} & \delta_{11}w_{22} + \delta_{12}w_{21} + \delta_{21}w_{12} + \delta_{22}w_{11} & \delta_{12}w_{22} + \delta_{22}w_{12}\\ \delta_{21}w_{21} & \delta_{21}w_{22} + \delta_{22}w_{21} & \delta_{22}w_{22} \end{bmatrix}$
其计算过程如下图所示（其中不同颜色块对应位置的值相加）：

另外，可以用卷积的形式计算 $x$的梯度：
$\frac{\partial L}{\partial x} = \left( \begin{array}{ccc} 0&0&0&0 \\ 0&\delta_{11}& \delta_{12}&0 \\ 0&\delta_{21}&\delta_{22}&0 \\ 0&0&0&0 \end{array} \right) * \left( \begin{array}{ccc} w_{22}&w_{21}\\ w_{12}&w_{11} \end{array} \right)$
此时卷积核 $w$需要旋转180度，也即上下翻转一次，然后左右翻转一次。输入矩阵需要增加padding。

设正向卷积的padding为 $p$，反向卷积的padding为 $p'$，则需要增加的padding为：
$p' = k-p-1$

其中 $k$是卷积核的大小，另外假设正向卷积的stride为1。