17.5 池化的前向计算与反向传播

17.5 池化层⚓︎

17.5.1 常用池化方法⚓︎

• 最大值池化，是取当前池化视野中所有元素的最大值，输出到下一层特征图中。
• 平均值池化，是取当前池化视野中所有元素的平均值，输出到下一层特征图中。

• 扩大视野：就如同先从近处看一张图片，然后离远一些再看同一张图片，有些细节就会被忽略
• 降维：在保留图片局部特征的前提下，使得图片更小，更易于计算
• 平移不变性，轻微扰动不会影响输出：比如上图中最大值池化的4，即使向右偏一个像素，其输出值仍为4
• 维持同尺寸图片，便于后端处理：假设输入的图片不是一样大小的，就需要用池化来转换成同尺寸图片

17.5.2 池化的其它方式⚓︎

\begin{cases} W_2 = (W_1 - F)/S + 1 \\\\ H_2 = (H_1 - F)/S + 1 \\\\ D_2 = D_1 \end{cases}

17.5.3 池化层的训练⚓︎

• 对于最大值池化，残差值会回传到当初最大值的位置上，而其它三个位置的残差都是0。
• 对于平均值池化，残差值会平均到原始的4个位置上。

Max Pooling⚓︎

w = max(a,b,e,f)

{\partial w \over \partial a} = 0, \quad {\partial w \over \partial b} = 1
{\partial w \over \partial e} = 0, \quad {\partial w \over \partial f} = 0

\delta_a = {\partial J \over \partial a} = {\partial J \over \partial w} {\partial w \over \partial a} = 0
\delta_b = {\partial J \over \partial b} = {\partial J \over \partial w} {\partial w \over \partial b} = \delta_w \cdot 1 = \delta_w
\delta_e = {\partial J \over \partial e} = {\partial J \over \partial w} {\partial w \over \partial e} = 0
\delta_f = {\partial J \over \partial f} = {\partial J \over \partial w} {\partial w \over \partial f} = 0

Mean Pooling⚓︎

w = \frac{1}{4}(a+b+e+f)

{\partial w \over \partial a} = \frac{1}{4}, \quad {\partial w \over \partial b} = \frac{1}{4}
{\partial w \over \partial e} = \frac{1}{4}, \quad {\partial w \over \partial f} = \frac{1}{4}

\delta_a = {\partial J \over \partial a} = {\partial J \over \partial w} {\partial w \over \partial a} = \frac{1}{4}\delta_w
\delta_b = {\partial J \over \partial b} = {\partial J \over \partial w} {\partial w \over \partial b} = \frac{1}{4}\delta_w
\delta_e = {\partial J \over \partial e} = {\partial J \over \partial w} {\partial w \over \partial e} = \frac{1}{4}\delta_w
\delta_f = {\partial J \over \partial f} = {\partial J \over \partial w} {\partial w \over \partial f} = \frac{1}{4}\delta_w

17.5.4 实现方法1⚓︎

class PoolingLayer(CLayer):
def forward_numba(self, x, train=True):
......

def backward_numba(self, delta_in, layer_idx):
......


@nb.jit(nopython=True)
def jit_maxpool_forward(...):
...
return z

@nb.jit(nopython=True)
def jit_maxpool_backward(...):
...
return delta_out


17.5.5 实现方法2⚓︎

class PoolingLayer(CLayer):
def forward_img2col(self, x, train=True):
......

def backward_col2img(self, delta_in, layer_idx):
......


17.5.6 性能测试⚓︎

Elapsed of numba: 17.537396907806396
Elapsed of img2col: 22.51519775390625
forward: True
backward: True


numba方法用了17秒，img2col方法用了22秒。并且两种方法的返回矩阵值是一样的，说明代码实现正确。

ch17, Level5