神经网络知识点1 - BP反向传播

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/a786150017/article/details/82953158

BP反向传播

基本原理

利用输出后的误差来估计输出层前一层的误差，再用这个误差估计更前一层的误差，如此一层一层地反传下去，从而获得所有其他各层的误差

对网络的连接权重做动态调整
核心：梯度下降法

推导过程

输入层相关变量：下标i
隐藏层相关变量：下标h
输出层相关变量：下标j
激励函数输入为a, 激励函数输出为z, 结点误差为δ
预测值是z, 目标值是t

【前向传播】
${a_h} = \sum\limits_i {{w_{ih}}{x_i} + {\theta _h}} \quad\quad{z_h} = f\left( {{a_h}} \right)$
${a_j} = \sum\limits_h {{w_{hj}}{z_h} + {\theta _j}} \quad\quad{z_j} = f\left( {{a_j}} \right)$

损失函数
$E\left( W \right) = {1 \over 2}\sum\limits_j {{{\left( {{t_j} - {z_j}} \right)}^2}}$
【反向传播】

链式法则，误差 × 输入

隐藏层-输出层
${{\partial E} \over {\partial {w_{hj}}}} = {\delta _j}{z_h}$
${{\partial E} \over {\partial {\theta _j}}} = {\delta _j}$
${\delta _j} = {{\partial E} \over {\partial {a_j}}} = {{\partial E} \over {\partial {z_j}}}{{\partial {z_j}} \over {\partial {a_j}}} = - \left( {{t_j} - {z_j}} \right)f'\left( {{a_j}} \right)$
输入层-隐藏层
${{\partial E} \over {\partial {w_{ih}}}} = {\delta _h}{x_i}$
${{\partial E} \over {\partial {\theta _h}}} = {\delta _h}$
${\delta _h} = {{\partial E} \over {\partial {a_h}}} = \sum\limits_j {{{\partial E} \over {\partial {a_j}}}{{\partial {a_j}} \over {\partial {z_h}}}{{\partial {z_h}} \over {\partial {a_h}}}} = \sum\limits_j {{\delta _j}{w_{hj}}f'\left( {{a_h}} \right)}$
【更新权重】

$\Delta w = \alpha {{\partial E} \over {\partial w}}\quad\quad{w^{t + 1}} = {w^t} - \Delta w$

$\Delta \theta = \alpha {{\partial E} \over {\partial \theta }}\quad\quad{\theta ^{t + 1}} = {\theta ^t} - \Delta \theta$