实数对向量求导公式，得到结果的形式与 分母/自变量 一致。
因变量是否转置对于结果无影响，这一条是我自己总结的。
公式一：
| $$ \nabla_{\mathbf{x}} (\mathbf{a}^T \mathbf{x}) = \color{red}{\nabla_{\mathbf{x}} (\mathbf{a}^T \mathbf{x})^T} = \nabla_{\mathbf x} (\mathbf{x}^T \mathbf{a}) = \color{blue}{\mathbf{a}} $$ |
| $$ \frac {\partial \mathbf{a}^T \mathbf{x}} {\partial \mathbf{x}} = \color{red}{\frac {\partial (\mathbf{a}^T \mathbf{x})^T} {\partial \mathbf{x}}} = \frac {\partial \mathbf{x}^T \mathbf{a}} {\partial \mathbf{x}} = \color{blue}{\mathbf{a}} $$ |
公式二：
| $$ \nabla_{\mathbf{x}} ||\mathbf{x}||_2^2 = \color{red} {\nabla_{\mathbf x} (\mathbf{x}^T \mathbf{x})} = \color{blue} {2 \mathbf{x}} $$ |
| $$ \frac {\partial ||\mathbf{x}||_2^2} {\partial \mathbf{x}} = \color{red} {\frac {\partial (\mathbf{x}^T \mathbf{x})} {\partial \mathbf{x}}} = \color{blue} { 2 \mathbf{x}} $$ |
公式三：
| $$ \nabla_{\mathbf x} (\mathbf{x}^T \mathbf{A} \mathbf{x}) = \color{red}{\nabla_{\mathbf x} (\mathbf{x}^T \mathbf{A} \mathbf{x})^T} = \nabla_{\mathbf x} (\mathbf{x}^T \mathbf{A}^T \mathbf{x}) = \color{blue} {(\mathbf{A} + \mathbf{A}^T) \mathbf{x}} $$ |
| $$ \frac {\partial (\mathbf{x}^T \mathbf{A} \mathbf{x})} {\partial \mathbf{x}} = \color{red}{\frac {\partial (\mathbf{x}^T \mathbf{A} \mathbf{x})^T} {\partial \mathbf{x}}} = \frac {\partial (\mathbf{x}^T \mathbf{A}^T \mathbf{x})} {\partial \mathbf{x}} = \color{blue} {(\mathbf{A} + \mathbf{A}^T) \mathbf{x}} $$ |