Does PyTorch loss() and backpropagation understand lambda layers?
I've been working with a resnet56 model from the code provided here: https://github.com/akamaster/pytorch_resnet_cifar10/blob/master/resnet.py. 

I noticed that the implementation is different from many of the other available ResNet examples online, and I was wondering if PyTorch's backpropagation algorithm using loss() can account for the lambda layer and shortcut in the code provided. 

If that is the case, can anyone provide insight into how PyTorch is able to interpret the lambda layer for backpropagation (i.e. how does PyTorch know how to differentiate with respect to the layer's operations)?

P.S. I also had to modify the code to fit my own use-case, and it seems like my own implementation with option == 'A' does not produce great results. This may simply be because option == 'B,' which uses convolutional layers instead of padding, is better for my data.

        self.shortcut = nn.Sequential()
        if stride != 1 or in_planes != planes:
            if option == 'A':
                top = (int) ((self.expansion*planes - in_planes) / 2)
                bot = (self.expansion*planes - in_planes) - top
                self.shortcut = LambdaLayer(lambda x:
                                            F.pad(x[:, :, ::stride, ::stride], (0, 0, 0, 0, top, bot), "constant", 0))


"I was wondering if PyTorch's backpropagation algorithm using loss() can account for the lambda layer and shortcut in the code provided."

PyTorch has no problem with backpropagating through lambda functions. Your LambdaLayer is just defining the forward pass of the Module as the evaluation of the lambda function, so your question boils down to whether PyTorch can backpropagate through lambda functions. 

"If that is the case, can anyone provide insight into how PyTorch is able to interpret the lambda layer for backpropagation (i.e. how does PyTorch know how to differentiate with respect to the layer's operations)?"

The lambda function performs the torch.nn.functional.Pad function on x, which we can packpropagate through because it is has a defined backwards() function. 

PyTorch handles lambda functions the same way an autodiff tool like PyTorch handles any function: it breaks it up into primitive operations, and uses the differentiation rules for each primitive operation to build up the derivative of the entire computation. 
