When retraining a model in pytorch should the optimizer be defined outside the train method? Will it get rid of previous weights if not?
So I am training a GNN on pytorch and after training it, with it's saved weights, I want to train it more with a separate dataset. When re training with the new dataset I don't want the weights to be reset, I want the weights to update from my last training session. Currently, my training code looks like this:
def train(data,model):
    train_loader, val_loader, test_loader, feature_len = data
    optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
    loss_fn = torch.nn.MSELoss()
    epoch = 17

    print('start training\n')
    
    evaluate(model, 'train', train_loader)
    evaluate(model, 'val', val_loader)
    evaluate(model, 'test', test_loader)

    for i in range(epoch):
        print('epoch %d:' % i)

        model.train()
        for graph1, graph2, target in train_loader:
            pred = torch.squeeze(model(graph1, graph2))
            loss = loss_fn(pred, target)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()

        
        evaluate(model, 'train', train_loader)
        evaluate(model, 'val', val_loader)
        evaluate(model, 'test', test_loader)
        print()

At the moment, I create my model object outside of the function, and then train it using that code above(I also have an evaluate function but it is left out so to be more specific with my question). My question is, if after using this train method, I decide to train again on more data, will the fact I have the optimizer definition within the method mean it will train from scratch again? If so, to avoid this would I just define my optimizer outside of the train method? I'm slightly confused about retraining my model with saved weights, the pytorch tutorials didn't help.

You can define the optimizer and model wherever you want (both inside and outside the train() method) as long as you are loading the weights correctly before the training loop. What you are missing probably is loading weights!!
From Pytorch tutorial,
Defining model and optimizer:
model = TheModelClass(*args, **kwargs)
optimizer = TheOptimizerClass(*args, **kwargs)

Loading the weights:
checkpoint = torch.load(PATH)
model.load_state_dict(checkpoint['model_state_dict'])
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
epoch = checkpoint['epoch']
loss = checkpoint['loss']

model.eval()
# - or -
model.train()

