Layer factory function to create an instance of a fully-connected linear layer of the form activation(input @ W + b) with weights W and bias b, and activation and b being optional. shape may describe a tensor as well.
Dense(shape, activation = activation_identity, init = init_glorot_uniform(), input_rank = NULL, map_rank = NULL, bias = TRUE, init_bias = 0, name = "")
| shape | - list of ints representing tensor shape |
|---|---|
| activation | (Function) - optional activation Function |
| init | (scalar or matrix or initializer, defaults to init_glorot_uniform()) – initial value of weights W |
| bias | (bool) – whether to include bias |
| init_bias | (scalar or matrix or initializer, defaults to 0) – initial value of weights b |
| name | string (optional) the name of the Function instance in the network |
A Dense layer instance owns its parameter tensors W and b, and exposes them as attributes .W and .b.
The Dense layer can be applied to inputs that are tensors, not just vectors. This is useful, e.g., at the top of a image-processing cascade, where after many convolutions with padding and strides it is difficult to know the precise dimensions. For this case, CNTK has an extended definition of matrix product, in which the input tensor will be treated as if it had been automatically flattened. The weight matrix will be a tensor that reflects the “flattened” dimensions in its axes.
This behavior can be modified by telling CNTK either the number of axes that should not be projected (map_rank) or the rank of the input (input_rank). If neither is specified, all input dimensions are projected, as in the example above.