You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should two methods to the layer modules (e.g. nn.* and cudnn.* modules). All these modules interact with our tools through two main methods: forward(input) and backward(input, gradOutput), that perform the forward and backpropagation passes through the module, respectively. It would be nice to have two similar methods that also receive the input and output sizes, in the case that the different batch elements have different sizes. For instance, consider the LSTM layer from the cudnn package:
layer:forward_with_sizes(input, inputSizes), this would return the output tensor AND a tensor with the output sizes. If the sizes do not change, it can just return inputSizes. For instance, in the Maxpool layer, the output sizes would need to be computed according to the stride/size parameters of the pooling.
layer:backward_with_sizes(input, gradOutput, outputSizes), this would return the gradInput and gradInputSizes tensors, to be passed to the previous layers during backpropagation.
The text was updated successfully, but these errors were encountered:
We should two methods to the layer modules (e.g. nn.* and cudnn.* modules). All these modules interact with our tools through two main methods: forward(input) and backward(input, gradOutput), that perform the forward and backpropagation passes through the module, respectively. It would be nice to have two similar methods that also receive the input and output sizes, in the case that the different batch elements have different sizes. For instance, consider the LSTM layer from the cudnn package:
layer:forward_with_sizes(input, inputSizes), this would return the output tensor AND a tensor with the output sizes. If the sizes do not change, it can just return inputSizes. For instance, in the Maxpool layer, the output sizes would need to be computed according to the stride/size parameters of the pooling.
layer:backward_with_sizes(input, gradOutput, outputSizes), this would return the gradInput and gradInputSizes tensors, to be passed to the previous layers during backpropagation.
The text was updated successfully, but these errors were encountered: