# 07.3 线性多分类的工作原理

## 7.3 线性多分类原理⚓︎

### 7.3.1 多分类过程⚓︎

1. 线性计算
z_1 = x_1 w_{11} + x_2 w_{21} + b_1 \tag{1}z_2 = x_1 w_{12} + x_2 w_{22} + b_2 \tag{2}z_3 = x_1 w_{13} + x_2 w_{23} + b_3 \tag{3}
1. 分类计算
a_1=\frac{e^{z_1}}{\sum_i e^{z_i}}=\frac{e^{z_1}}{e^{z_1}+e^{z_2}+e^{z_3}} \tag{4}  a_2=\frac{e^{z_2}}{\sum_i e^{z_i}}=\frac{e^{z_2}}{e^{z_1}+e^{z_2}+e^{z_3}} \tag{5}  a_3=\frac{e^{z_3}}{\sum_i e^{z_i}}=\frac{e^{z_3}}{e^{z_1}+e^{z_2}+e^{z_3}} \tag{6}
1. 损失函数计算

\begin{aligned} loss(w,b)&=-(y_1 \ln a_1 + y_2 \ln a_2 + y_3 \ln a_3) \\\\ &=-\sum_{j=1}^{n} y_j \ln a_j \end{aligned} \tag{7}

\begin{aligned} J(w,b) &=- \sum_{i=1}^m (y_{i1} \ln a_{i1} + y_{i2} \ln a_{i2} + y_{i3} \ln a_{i3}) \\\\ &=- \sum_{i=1}^m \sum_{j=1}^n y_{ij} \ln a_{ij} \end{aligned} \tag{8}

### 7.3.2 数值计算举例⚓︎

z=[z_1,z_2,z_3]=[3,1,-3]

a=[a_1,a_2,a_3]=[0.879,0.119,0.002]

#### 如果标签值表明此样本为第一类⚓︎

y=[1,0,0]

loss_1=-(1 \times \ln 0.879 + 0 \times \ln 0.119 + 0 \times \ln 0.002)=0.123

a-y=[-0.121,0.119,0.002]

#### 如果标签值表明此样本为第二类⚓︎

y=[0,1,0]

loss_2=-(0 \times \ln 0.879 + 1 \times \ln 0.119 + 0 \times \ln 0.002)=2.128

a-y=[0.879,-0.881,0.002]

### 7.3.3 多分类的几何原理⚓︎

a_j = \frac{e^{z_j}}{\sum\limits_{i=1}^3 e^{z_i}}=\frac{e^{z_j}}{e^{z_1}+e^{z_2}+^{z_3}}

#### 当样本属于第一类时⚓︎

a_1 > a_2 且 a_1 > a_3 \tag{9}

z_1 > z_2, z_1 > z_3 \tag{10}

x_1 w_{11} + x_2 w_{21} + b_1 > x_1 w_{12} + x_2 w_{22} + b_2 \tag{11}x_1 w_{11} + x_2 w_{21} + b_1 > x_1 w_{13} + x_2 w_{23} + b_3 \tag{12}

(w_{21} - w_{22})x_2 > (w_{12} - w_{11})x_1 + (b_2 - b_1) \tag{13}
(w_{21} - w_{23})x_2 > (w_{13} - w_{11})x_1 + (b_3 - b_1) \tag{14}

w_{21} > w_{22}, w_{21}> w_{23} \tag{15}

x_2 > \frac{w_{12} - w_{11}}{w_{21} - w_{22}}x_1 + \frac{b_2 - b_1}{w_{21} - w_{22}} \tag{16}
x_2 > \frac{w_{13} - w_{11}}{w_{21} - w_{23}} x_1 + \frac{b_3 - b_1}{w_{21} - w_{23}} \tag{17}

y > W_{12} \cdot x + B_{12} \tag{18}
y > W_{13} \cdot x + B_{13} \tag{19}

#### 当样本属于第二类时⚓︎

z_2 > z_1,z_2 > z_3 \tag{20}

y < W_{12} \cdot x + B_{12} \tag{21}
y > W_{23} \cdot x + B_{23} \tag{22}

#### 当样本属于第三类时⚓︎

z_3 > z_1,z_3 > z_2 \tag{22}

y < W_{13} \cdot x + B_{13} \tag{23}
y < W_{23} \cdot x + B_{23} \tag{24}

### 思考与练习⚓︎

1. 我们假设 $w_{21} > w_{22} > w_{23}$ 是否有根据呢？假设 $w_{22} > w_{21} > w_{23}$，直线的位置会有所变化吗？
2. 最后一张图的三条直线应该相交于一点吗？