Should I run a separate thread to save the model in Pytorch
Consider the following snippet from PyTorch Imagenet Example (Please notice the "#AREA OF INTEREST"):

try:
for epoch in range(1, args.epochs+1):
    epoch_start_time = time.time()
    train()
    val_loss = evaluate(val_data)
    # Save the model if the validation loss is the best we've seen so far.
    if not best_val_loss or val_loss &lt; best_val_loss:
        # AREA OF INTEREST ##########
        with open(args.save, 'wb') as f:
            torch.save(model, f)
        #############################
        best_val_loss = val_loss
    else:
        # Anneal the learning rate if no improvement has been seen in the validation dataset.
        lr /= 4.0


The question is: should I run a separate thread that saves the model in order to save time, or am I overkilling the process? i.e., it does not worth running a separate thread? I've checked the doc of torch.save but I did not find what I want.

It depends on your model size. If you have slow IO and big model it may take time. But usual FS cache is big enough to store a whole model.
