朴素贝叶斯（Simple Example）

1 假设

计算 $P(X|C_i)$ ,朴素贝叶斯分类假设类条件独立，即给定样本属性值相互条件独立。

P (x_{1}, \dots, x_{k} | C_{i}) = P (x_{1} | C_{i}) \cdot \dots \cdot P (x_{k} | C_{i})

$P(x_1,…,x_k|C_i) = P(x_1|C_i)·…·P(x_k|C_i)$

2 Notion

贝叶斯定理：

P (C_{i} | X) = \frac{P (X | C_{i}) \cdot P (C_{i})}{P (X)} = \frac{P (X | C_{i}) \cdot P (C_{i})}{\sum_{j = 1}^{c} P (X | C_{j}) \cdot P (C_{j})}

$P(C_i|X)= \frac {P(X|C_i) \ \cdot \ P(C_i)} {P(X)} = \frac {P(X|C_i) \ \cdot \ P(C_i)} {\sum_{j=1}^{c}P(X|C_j) \ \cdot \ P(C_j) }$

$i$ 表示 $label$ 的类别数

$j$ 也表示 $label$ 的类别数，只是为了区别于 $i$

先验概率 $prior \ probability$ ： $P(C_i)$

概率密度函数 $probability \ density \ function$ : $P(X|C_i)$

后验概率 $posteriori \ probabilities$ ： $P(C_i|X)$

总结，根据先验概率和概率密度函数，计算后验概率

eg: 对于一个二分类问题， $yes \ or \ no$ ，对应的贝叶斯公式如下

P (Y e s | X) = \frac{P (X | Y e s) \cdot P (Y e s)}{P (X)} = \frac{P (X | Y e s) \cdot P (Y e s)}{P (X | Y e s) \cdot P (Y e s) + P (X | N o) \cdot P (N o)}

$P(Yes|X)= \frac {P(X|Yes) \cdot P(Yes)} {P(X)} = \frac {P(X|Yes) \ \cdot \ P(Yes)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)}$

P (N o | X) = \frac{P (X | N o) \cdot P (N o)}{P (X)} = \frac{P (X | N o) \cdot P (N o)}{P (X | Y e s) \cdot P (Y e s) + P (X | N o) \cdot P (N o)}

$P(No|X)= \frac {P(X|No) \cdot P(No)} {P(X)} = \frac {P(X|No) \ \cdot \ P(No)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)}$

如果 $P(Yes|X)>P(No|X)$ ，分类结果为 $Yes$ ，反之结果为 $No$

3 Simple Example

对 $X = \left \{ Gender = Female,Income = High, Age = Middle \right \}$ 计算分类结果 $Yes \ or \ No$

$P(Yes) = 3 / 6$

由图知

$P\left (Gender = Female \mid Yes\right ) = 2/3$

$P\left ( Income = High \mid Yes\right ) = 3/3$

$P\left ( Age = Middle \mid Yes\right ) = 1/3$

所以

P (X ∣ Y e s) \cdot P (Y e s) = P (G e n d e r = F e m a l e ∣ Y e s) \cdot P (I n c o m e = H i g h ∣ Y e s) \cdot P (A g e = M i d d l e ∣ Y e s) \cdot P (Y e s) = \frac{2}{3} \times \frac{3}{3} \times \frac{1}{3} \times \frac{3}{6} \approx 0.111

$P\left ( X\mid Yes \right ) \cdot P(Yes)= P\left (Gender = Female \mid Yes\right )\cdot P\left ( Income = High \mid Yes\right )\cdot P\left ( Age = Middle \mid Yes\right ) \cdot P(Yes) = \frac{2}{3}\times\frac{3}{3}\times\frac{1}{3}\times\frac{3}{6} \approx 0.111$

$P(No) = 3/6$

由图知

$P\left (Gender = Female \mid No\right ) = 1/3$

$P\left ( Income = High \mid No\right ) = 1/3$

$P\left ( Age = Middle \mid No\right ) = 2/3$

所以

P (X ∣ N o) \cdot P (N o) = P (G e n d e r = F e m a l e ∣ N o) \cdot P (I n c o m e = H i g h ∣ N o) \cdot P (A g e = M i d d l e ∣ N o) \cdot P (N o) = \frac{1}{3} \times \frac{1}{3} \times \frac{2}{3} \times \frac{3}{6} = 0.037

$P\left ( X\mid No \right ) \cdot P(No)= P\left (Gender = Female \mid No\right )\cdot P\left ( Income = High \mid No\right )\cdot P\left ( Age = Middle \mid No\right ) \cdot P(No) = \frac{1}{3}\times\frac{1}{3}\times\frac{2}{3}\times\frac{3}{6} = 0.037$

P (Y e s | X) = \frac{P (X | Y e s) \cdot P (Y e s)}{P (X | Y e s) \cdot P (Y e s) + P (X | N o) \cdot P (N o)} = \frac{0.111}{0.111 + 0.037} = 75 %

$P(Yes|X)= \frac {P(X|Yes) \ \cdot \ P(Yes)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)} = \frac{0.111}{0.111+0.037} = 75 \%$

P (N o | X) = \frac{P (X | N o) \cdot P (N o)}{P (X | Y e s) \cdot P (Y e s) + P (X | N o) \cdot P (N o)} = \frac{0.037}{0.111 + 0.037} = 25 %

$P(No|X)= \frac {P(X|No) \ \cdot \ P(No)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)} = \frac{0.037}{0.111+0.037} = 25 \%$

因为

$P(Yes|X)>P(No|X)$

所以

分类结果为 $Yes$