AdaBoost实例

试用AdaBoost算法学习一个强分类器

训练数据集

序号	1	2	3	4	5	6	7	8	9	10
x	0	1	2	3	4	5	6	7	8	9
y	1	1	1	-1	-1	-1	1	1	1	-1

解：
初始化数据权值分布
$D_1=(w_{1,1},w_{1,2},\dots,w_{1,10})\\ w_{1,i}=0.1,i=1,2,\dots,10$
对于 $m=1$,
  (a)在权值分布为 $D_1$的训练数据上，计算阈值 $\nu$取[0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5]时分类误差率，

序号	1	2	3	4	5	6	7	8	9
$\nu$	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5
分类误差率	0.5	0.4	0.3	0.4	0.5	0.4	0.5	0.4	0.3

阈值取 $\nu=8.5$时分类误差率最低，故基本分类器为
$G_1(x)= \begin{cases} 1,&x\lt8.5 \\ -1,&x\ge8.5 \end{cases}$
  (b) $G_1(x)$在训练数据集上的误差率 $e_1=P(G_1(x_i)\neq y_i) =0.3$
  ©计算 $G_1(x)$的系数: $\alpha_1=\dfrac{1}{2}log\dfrac{1-e_1}{e_1}=0.4236$
  (d)更新训练数据的权值分布：
$D_2=(w_{2,1},w_{2,2},\dots,w_{2,10})$
$w_{2,i} = \dfrac{w_{1,i}}{Z_1}exp(-\alpha_1y_iG_1(x_i)),i=1,2,\dots,10$
$D_2=(0.07142857,0.07142857,0.07142857,0.16666667,0.16666667,0.16666667,0.07142857,0.07142857,0.07142857,0.07142857)$
$f_1(x)=\alpha_1G_1(x)=0.4236G_1(x)$
  (e)分类器 $sign[f_1(x)]$在训练数据集上有3个误分点

序号	1	2	3	4	5	6	7	8	9	10
$G_1(x)$	1	1	1	1	1	1	1	1	1	-1
$f_1(x)$	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	-0.4236
$sign[f_1(x)]$	1	1	1	1	1	1	1	1	1	-1
y	1	1	1	-1	-1	-1	1	1	1	-1

对 $m=2$,
  (a)在权值分布为 $D_2$的训练数据上，计算阈值 $\nu$取[0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5]时分类误差率， $e_m=\sum_{G_m(x_i)\neq y_i} w_{mi}$

序号	1	2	3	4	5	6	7	8	9
$\nu$	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5
分类误差率	0.357	0.286	0.214	0.381	0.452	0.286	0.358	0.429	0.5

阈值取 $\nu=2.5$时分类误差率最低，故基本分类器为
$G_2(x)= \begin{cases} 1,&x\lt2.5 \\ -1,&x\ge2.5 \end{cases}$
  (b) $G_2(x)$在训练数据集上的误差率 $e_2=P(G_2(x_i)\neq y_i) =0.214$
  ©计算 $G_2(x)$的系数: $\alpha_2=\dfrac{1}{2}log\dfrac{1-e_2}{e_2}=0.6496$
  (d)更新训练数据的权值分布：
$D_3=(w_{3,1},w_{3,2},\dots,w_{3,10})$
$w_{3,i} = \dfrac{w_{2,i}}{Z_1}exp(-\alpha_2y_iG_2(x_i)),i=1,2,\dots,10$
$D_3=(0.04545452,0.04545452,0.04545452,0.10606056,0.10606056,0.10606056, 0.16666675,0.16666675,0.16666675,0.04545452)$
$f_2(x)=0.4236G_1(x) + 0.6496G_2(x)$
  (e)分类器 $sign[f_2(x)]$在训练数据集上有3个误分点

序号	1	2	3	4	5	6	7	8	9	10
$G_1(x)$	1	1	1	1	1	1	1	1	1	-1
$G_2(x)$	1	1	1	-1	-1	-1	-1	-1	-1	-1
$\alpha_1G_1(x)$	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	-0.4236
$\alpha_2G_2(x)$	0.6496	0.6496	0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496
$sign[f_2(x)]$	1	1	1	-1	-1	-1	-1	-1	-1	-1
y	1	1	1	-1	-1	-1	1	1	1	-1

对 $m=3$
  (a)在权值分布为 $D_3$的训练数据上，计算阈值 $\nu$取[0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5]时分类误差率， $e_m=\sum_{G_m(x_i)\neq y_i} w_{mi}$

序号	1	2	3	4	5	6	7	8	9
$\nu$	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5
分类误差率	0.409	0.455	0.5	0.394	0.288	0.182	0.348	0.485	0.318

阈值取 $\nu=5.5$时分类误差率最低，故基本分类器为
$G_2(x)= \begin{cases} -1,&x\lt5.5 \\ 1,&x\ge5.5 \end{cases}$
  (b) $G_3(x)$在训练数据集上的误差率 $e_3=P(G_3(x_i)\neq y_i) =0.7520$
  (d)更新训练数据的权值分布：
$D_4=(w_{4,1},w_{4,2},\dots,w_{4,10})$
$w_{4,i} = \dfrac{w_{3,i}}{Z_1}exp(-\alpha_3y_iG_3(x_i)),i=1,2,\dots,10$
$D_4=(0.125,0.125,0.125,0.06481478,0.06481478,0.06481478, 0.10185189,0.10185189,0.10185189,0.125)$
$f_3(x)=0.4236G_1(x) + 0.6496G_2(x)+0.7520G_3(x)$
  (e)分类器 $sign[f_3(x)]$在训练数据集上有0个误分点

序号	1	2	3	4	5	6	7	8	9	10
$G_1(x)$	1	1	1	1	1	1	1	1	1	-1
$G_2(x)$	1	1	1	-1	-1	-1	-1	-1	-1	-1
$G_3(x)$	-1	-1	-1	-1	-1	-1	1	1	1	1
$\alpha_1G_1(x)$	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	0.4236	-0.4236
$\alpha_2G_2(x)$	0.6496	0.6496	0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496	-0.6496
$\alpha_3G_3(x)$	-0.7520	-0.7520	-0.7520	-0.7520	-0.7520	-0.7520	0.7520	0.7520	0.7520	0.7520
$sign[f_3(x)]$	1	1	1	-1	-1	-1	1	1	1	-1
y	1	1	1	-1	-1	-1	1	1	1	-1

于是最终的分类器为
$G(x)=sign[f_3(x)]=0.4236G_1(x) + 0.6496G_2(x)+0.7520G_3(x)$