矩阵求导
在看多元线性回归的闭式解的时候遇到矩阵求导问题，总体来讲矩阵求导与函数求导有极大的相似性，查看wiki后记录下矩阵求导的一些性质，方面日后查看。
用到比较多的公式如下(分母布局)：
\[ \frac{\partial \boldsymbol A\boldsymbol x}{\partial \boldsymbol x}=A^T \]
\[ \frac{\partial \boldsymbol x^T\boldsymbol A}{\partial \boldsymbol x}=A \]
\[ \frac{\partial \boldsymbol u \cdot \boldsymbol v}{\partial \boldsymbol x}=\frac{\partial \boldsymbol u^T\boldsymbol v}{\partial \boldsymbol x}=\frac{\partial \boldsymbol u}{\partial \boldsymbol x}\boldsymbol v+\frac{\partial \boldsymbol v}{\partial \boldsymbol x}\boldsymbol u \]
证明几个公式：
1.\(\cfrac{\partial \boldsymbol x^T\boldsymbol A \boldsymbol x}{\partial \boldsymbol x}=(\boldsymbol A+\boldsymbol A^T)\boldsymbol x\)
\[ \begin{align} \frac{\partial \boldsymbol x^T\boldsymbol A \boldsymbol x}{\partial \boldsymbol x}=&\frac{\partial (\boldsymbol A^T \boldsymbol x)^T \boldsymbol x}{\partial \boldsymbol x}\ \\=&\frac{\partial \boldsymbol A^T \boldsymbol x}{\partial \boldsymbol x}\boldsymbol x+\frac{\partial \boldsymbol x}{\partial \boldsymbol x}\boldsymbol A^T \boldsymbol x\\ =&\boldsymbol A\boldsymbol x+\boldsymbol A^T\boldsymbol x\\ =&(\boldsymbol A+\boldsymbol A^T)\boldsymbol x \end{align} \]
2.$ \cfrac{\partial E_{\hat{\boldsymbol w}}}{\partial \hat{\boldsymbol w}}=2\mathbf{X}^T(\mathbf{X}\hat{\boldsymbol w}-\boldsymbol{y})$
\[ \begin{align} \cfrac{\partial E_{\hat{\boldsymbol w}}}{\partial \hat{\boldsymbol w}}=& \cfrac{\partial \boldsymbol{y}^T\boldsymbol{y}}{\partial \hat{\boldsymbol w}}-\cfrac{\partial \boldsymbol{y}^T\mathbf{X}\hat{\boldsymbol w}}{\partial \hat{\boldsymbol w}}-\cfrac{\partial \hat{\boldsymbol w}^T\mathbf{X}^T\boldsymbol{y}}{\partial \hat{\boldsymbol w}}+\cfrac{\partial \hat{\boldsymbol w}^T\mathbf{X}^T\mathbf{X}\hat{\boldsymbol w}}{\partial \hat{\boldsymbol w}}\\ =&0-(\boldsymbol{y}^T\mathbf{X})^T-\mathbf{X}^T\boldsymbol{y}+(\mathbf{X}^T\mathbf{X}+\mathbf{X}^T\mathbf{X})\hat{\boldsymbol w}\\ =&2\mathbf{X}^T(\mathbf{X}\hat{\boldsymbol w}-\boldsymbol{y}) \end{align} \]
参考文献：
https://en.wikipedia.org/wiki/Matrix_calculus#Scalar-by-vector_identities