Layer factory function to create a function that implements layer normalization.
LayerNormalization(initial_scale = 1, initial_bias = 0, epsilon = 1e-05, name = "")
| epsilon | (float, default 0.00001) - added to avoid division by 0 |
|---|---|
| name | string (optional) the name of the Function instance in the network |
| init | (scalar or matrix or initializer, defaults to init_glorot_uniform()) – initial value of weights Wial_scale |
| init_bias | (scalar or matrix or initializer, defaults to 0) – initial value of weights b |
Layer normalization applies this formula to every input element (element-wise): y = (x - mean(x)) / (stddev(x) + epsilon) * scale + bias where scale and bias are learned scalar parameters.