Matrix inner product derivation/matrix derivation including Hadamard root/matrix elements-wise square root/matrix element-wise square root derivation/F norm derivation

There are relatively few matrix derivation cases including Hadamard root, this case is for reference only:

---

1 topic

Given $\mathbf{X}\in {\mathbb{R}}^n$ ,$\mathbf{A}\in R^{n\times n}$ ,$f(\mathbf{X})={\sum }_{i=1}^n\sqrt{|\mathbf{A}\mathbf{X}{|}_i^2+d^2}$. where $\sqrt{(\cdot )}$Represents the Hadamard root (elements-wise square root), that is, the square root of the matrix elements item by item. Find $f^{\prime }(\mathbf{X})$，即$\frac{\partial f}{\partial \mathbf{X}}$。

2 solve

2.1 First use the Hadamard product to solve the square root

令$$\mathbf{v}=\sqrt{|\mathbf{A}\mathbf{X}{|}^2+d^{2}1}$$

$$\begin{array}{rl}\therefore v\odot v& =|AX{|}^2+d^{2}1\\ & =\mathbf{A}\mathbf{X}\odot AX+d^{2}1\end{array}$$

According to the property of differential Hadamard product : $d(\mathbf{X}\odot \mathbf{Y})=\mathbf{X}\odot d\mathbf{Y}+d\mathbf{X}\odot Y$有:
$$\begin{array}{rl}d(\mathbf{v}\odot \mathbf{v})& =\mathbf{v}\odot d\mathbf{v}+dv\odot v\\ & =v\odot dv+v\odot dv\\ & =2v\odot d\mathbf{v}\end{array}$$

即:
$$\begin{array}{rl}2v\odot d\mathbf{v}& =d\left(\mathbf{A}\mathbf{X}\odot \mathbf{A}\mathbf{X}+d^{2}1\right)\\ & =d(\mathbf{A}\mathbf{X}\odot \mathbf{A}\mathbf{X})+d(d^{2}1)\\ & =2\mathbf{A}\mathbf{X}\odot d(\mathbf{A}\mathbf{X})\\ & =2AX\odot \left(\right(dA)X+\mathbf{A}d\mathbf{X})\\ & =2AX\odot \mathbf{A}d\mathbf{X}\end{array}$$

$$\therefore d\mathbf{v}=\mathbf{A}\mathbf{X}\odot \mathbf{A}d\mathbf{X}\oslash v$$
where$\oslash$ is Hadamard division / elements-wise division, that is, matrix item-wise division, and$\odot$ have similar properties. Or let$\mathbf{p}\odot \mathbf{v}=1$，即$p$ is$vBy$ Hadamard inverse / element-wise inverse, equation$d\mathbf{v}=\mathbf{A}\mathbf{X}\odot \mathbf{A}d\mathbf{X}\odot \mathbf{p}$.

2.2 Using the properties of Frobenius inner product (matrix inner product) and trace, the solution can be obtained

Give the Frobenius inner product : $\mathbf{A}:\mathbf{B}=\mathrm{tr}({\mathbf{A}}^{T}B)$, can get:
$$\begin{array}{rl}f& =1:\mathbf{v}\\ \therefore df& =d(1:\mathbf{v})\\ & =d1:v+1:d\mathbf{v}(nature:\nabla (\mathbf{A}:\mathbf{B})=\nabla \mathbf{A}:\mathbf{B}+\mathbf{A}:\nabla \mathbf{B})\\ & =1:dv\\ & =1:(\mathbf{A}\mathbf{X}\odot \mathbf{A}d\mathbf{X}\oslash v)\\ & =1:(AX\oslash v\odot \mathbf{A}d\mathbf{X})(nature:\mathbf{X}\odot \mathbf{Y}=Y\odot \mathbf{X})\\ & =(1\odot (AX\oslash \mathbf{v})):(\mathbf{A}d\mathbf{X})(nature:\mathbf{C}:(\mathbf{A}\odot B)=(\mathbf{C}\odot \mathbf{A}):B)\\ & =(AX\oslash v):\left(AdX\right)\\ & =A^T(\mathbf{A}\mathbf{X}\oslash v):d\mathbf{X}(nature:\mathbf{C}\mathbf{A}:B=A:C^T\mathbf{B}=C:\mathbf{B}{\mathbf{A}}^T\\ & =\mathrm{tr}(({\mathbf{A}}^T(\mathbf{A}\mathbf{X}\oslash \mathbf{v}){)}^{T}dX\left)\right(Definition\; ofmatrixinnerproduct)\\ Right\; now\frac{\partial f}{\partial \mathbf{X}}& ={\mathbf{A}}^T(AX\oslash \mathbf{v})(properties:df=\mathrm{tr}\left({\left(\frac{\partial f}{\partial \mathbf{X}}\right)}^{T}d\mathbf{X}\right))\end{array}$$
If the Hadamard inverse defined above is used, the result can also be expressed as:
$$\frac{\partial f}{\partial X}=A^T(AX\odot \mathbf{p})$$

Summary:
First use the Hadamard product to solve the square root, and then use the correlation properties to get $dv$ , and finally use the properties of matrix inner product and trace to get the solution.

Reference link:

• Partial Derivative of Square Root with Matrix Multiplication
• How to compute the derivative of the following function with matrix X as the variable?