Modify PyTorch model for inference - then resume training
I’d like to alternate inference and training in a model in PyTorch, but I need to modify it during inference, and I have some questions about that.
Can I unload the model from the gpu by calling model.to('cpu'), make a modified copy (and run it on the gpu), and then move the original back to gpu by calling model.to('gpu')? In other words, is moving the model gpu-&gt;cpu-&gt;gpu a lossless operation? What happens to the parameters that were passed to the optimizer? I don’t want to lose the optimizer state
What is the best way to make a copy of an in-memory model? I can save it and then reload a copy, but not sure if that is necessary just to copy.
If I want to run inference in half precision (more than 2x faster in this case), can I change the model to half and then change it back? Is that lossless? (Does the model keep a full-precision copy of everything, or does it replace weights with half-precision in place?)
The model is a ResNet50 lookalike.  Not enough GPU memory for two models :)

&quot;What is the best way to make a copy of an in-memory model? I can save it and then reload a copy, but not sure if that is necessary just to copy.&quot;
You can copy a model with:
import copy
...
best_model = copy.deepcopy(model)

with best_model you can save to disk or load in other model, etc
