How to perform finetuning on a Pytorch net
I'm using this implementation of SegNet in Pytorch, and I want to finetune it.
I've read online and I've found this method (basically freezing all layers except the last one in your net). My problem is that SegNet has more than 100 layers and I'm looking for a simpler way to do it, rather than writing 100 lines of code. 

Do you think this could work? Or is this utter nonsense?

import torch.optim as optim

model = SegNet()
for name, param in model.named_modules():
    if name != 'conv11d':    # the last layer should remain active
        param.requires_grad = False

optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)


def train():
    ...


How can I check if this is working as intended?

This process is called finetuning and setting requires_grad to False is a good way to do this. From the pytorch docs:


  Every Tensor has a flag: requires_grad that allows for fine grained exclusion of subgraphs from gradient computation and can increase efficiency.


...


  If there’s a single input to an operation that requires gradient, its output will also require gradient. Conversely, only if all inputs don’t require gradient, the output also won’t require it. Backward computation is never performed in the subgraphs, where all Tensors didn’t require gradients.


See this pytorch tutorial for a relevant example.

One simple way of checking to see this is working is looking at the initial error rates. Assuming the task is similar to the task the net was originally trained on, they should be much lower than for a randomly initialized net.
