Machine learning principal component analysis

See: https://blog.csdn.net/program_developer/article/details/80632779

1. Overview 1. Concept:

"主成分分析"(Principal Component Analysis;PCA)的主要思想是将n维特征映射到k维上.这k维是全新的正交特征,称为"主成分",是在原有n
维特征的基础上重新构造出来的.形象地说,PCA就是在原特征空间中找到1组相互正交的轴.其中,第1个新轴是原始数据中方差最大的方向,第2个新轴
是与第1个坐标轴正交的平面中方差最大的方向...以此类推.计算方法为:计算数据矩阵的协方差矩阵,然后得到协方差矩阵的特征值和特征向量,选择
对应的特征值最大(即方差最大)的k个特征向量,由这些向量张成新的特征空间.由于有2种方法可以特征值和特征向量:特征分解和奇异值分解阵,所以
PCA有两种实现:"基于特征分解协方差矩阵的PCA"和"基于SVD分解协方差矩阵的PCA"

2. Function:

①降维,减少计算开销
②便于可视化
③去除噪声,避免过拟合
④发现更容易被人理解的特征

2. Realization
1. Principle:

PCA based on eigen decomposition covariance matrix:
① Let the original variables $X_1,X_2\dots X_p$The $The\; n$ observation data matrix is$$X=\left[\begin{array}{ccc}{x}_{11}& ...& x_{1W}\\ ...& ...& ...\\ x_{n1}& ...& x_{np}\end{array}\right]=\left[\begin{array}{c}{X}_1,X_2...X_p\end{array}\right]\left(1\right)$$
② Standardize the data matrix by column, and the result is still recorded as$X$
③Find the correlation coefficient matrix$R=(r_{ij}{)}_p$
④ Solve $R$ characteristic equation of$|R-\lambda E|=0$ , the characteristic root is${\lambda }_1\ge {\lambda }_2\ge ...\ge {\lambda }_p>0$
⑤Determine the number of principal components$m$,使$$\frac{\sum _{i=1}^m{\lambda }_i}{\sum _{i=1}^p{\lambda }_i}\ge \alpha$$ where$\alpha$ needs to be determined according to the actual situation, usually$80\%$
⑥ find the largestmmThe unit eigenvector corresponding to$m$ eigenvalues$$b_1=\left[\begin{array}{c}{b}_{11}\\ ...\\ b_{p1}\end{array}\right],b_2=\left[\begin{array}{c}{b}_{12}\\ ...\\ b_{p2}\end{array}\right]...b_1=\left[\begin{array}{c}{b}_{1m}\\ ...\\ b_{pm}\end{array}\right](2)$$
⑦求主成分$${WITH}_i=b_{1I}{X}_1+...+b_{pi}{X}_p(i=1,2...m)$$

2. Realization: