Pytorch resumes training after every training session
I have a dataset which is partitioned into smaller datasets. 

I want to train 3 models for each partition of the dataset, but I need all training sessions to start from the same initialised network parameters. 

so it looks like this:

modelList = []
thisCNN = NNet()

for x in range(3):
    train = torch.utils.data.DataLoader(Subset(train_set, indexes[x]), batch_size=32)
    bb = trainMyNet(thisCNN, train, test)
    modelList.append(list(bb.parameters()))

print modelList[0][1]
print modelList[1][1]
print modelList[2][1]


In the printing, I am getting the same exact parameters for every saved model which is strange, and also I noticed that after every iteration the model in fact resumes the training from the previous iteration as it results in lower loss per iteration. 

What I am trying to achieve, is, per iteration to get a new model on the current subset x but the training should start with the same initial thisCNN = NNet() weights. 

When you call bb = trainMyNet(thisCNN, train, test) you are not taking a copy of thisCNN, but it is the same model you are updating in each iteration. To get your code working you should probably pass a copy of this model:

from copy import deepcopy    
modelList = []
thisCNN = NNet()

for x in range(3):
    train = torch.utils.data.DataLoader(Subset(train_set, indexes[x]), batch_size=32)
    bb = trainMyNet(deepcopy(thisCNN), train, test)
    modelList.append(list(bb.parameters()))

print modelList[0][1]
print modelList[1][1]
print modelList[2][1]


This should initialize all models as thisCNN and make sure that they are different after training.
