Which seed when using `pytorch.manual_seed(seed)`?
I have trained a model with ImageNet. I got a new GPU and I also want to train the same model on a different GPU.
I want to compare if the outcome is different and therefore I want to use torch.manual_seed(seed).
After reading the docs https://pytorch.org/docs/stable/generated/torch.manual_seed.html it is still unclear, which number I should take for the paramater seed.
How to choose the seed parameter? Can I take 10, 100, or 1000 or even more or less and why?


How to choose the seed parameter? Can I take 10, 100, or 1000 or even more or less and why?

The PyTorch doc page you are pointing to does not mention anything special, beyond stating that the seed is a 64 bits integer.
So yes, 1000 is OK. As you expect from a modern pseudo-random number generator, the statistical properties of the pseudo-random sequence you are relying on do NOT depend on the choice of seed.
As for most runs you will probably reuse the seed from a previous run, a practical thing is to have the random seed as a command line parameter. In those cases where you have to come up with a brand new seed, you can peek one from the top of your head, or just play dice to get it.
The important thing is to have a record of the seed used for each run.
OK, but ...
That being said, a lot of people seem uncomfortable with the task of just “guessing” some arbitrary number. There are (at least) two common expedients to get some seed in a seemingly proper “random” fashion.
The first one is to use the operating system official source for genuinely (not pseudo) random bits. In Python, this is typically rendered as os.urandom(). So to get a seed in Python as a 64 bits integer, you could use code like this:
import  functools
import  os

# returns a list of 8 random small integers between 0 and 255
def get8RandomBytesFromOS():
    r8 = os.urandom(8)  # official OS entropy source
    byteCodes = list(map(ord, r8.decode('Latin-1')))  # type conversion
    return byteCodes

# make a single long integer from a list of 8 integers between 0 and 255
def getIntFromBytes(bs):
    # force highest bit to 0 to avoid overflow
    bs2 = [bs[0] if (bs[0] &lt; 128) else (bs[0]-128)]  +  bs[1:8]
    num = functools.reduce(lambda acc,n: acc*256+n, bs2)
    return num

# Main user entry point:
def getRandomSeedFromOS():
   rbs8 = get8RandomBytesFromOS()
   return (getIntFromBytes(rbs8))

A second common expedient is to hash a string containing the current date and time, possibly with some prefix. With Python, the time includes microseconds. When a human user launches a script, the microseconds number of the launch time can be said to be random. One can use code like this, using a version of SHA (Secure Hash Algorithm):
import  hashlib
import  datetime

def getRandomSeedFromTime():
    prefix      =  'dl.cs.univ-stackoverflow.edu'
    timeString1 =  str(datetime.datetime.now())
    timeString2 =  prefix + ' ' + timeString1
    hash        =  hashlib.sha256(timeString2.encode('ascii'))
    bytes       =  (hash.digest())[24:32]  #  8 rightmost bytes of hash
    byteCodes   =  list(map(ord, bytes.decode('Latin-1')))  # type conversion
    return (getIntFromBytes(byteCodes))

But, again, 1000 is basically OK. The idea of hashing the time string, instead of just taking the number of microseconds since the Epoch as the seed, probably comes from the fact that some early random number generators used their seed as an offset into a common and not so large global sequence. Hence, if your program naïvely took two seeds in rapid sequence without hashing, there was a risk of overlap between your two sequences. Fortunately, pseudo-random number generators now are much better than what they used to be.
(Addendum - taken from comment)
Note that the seed is a peripheral thing. The important thing is the state of the automaton, which can be much larger than the seed, hence the very word “seed”. The commonly used Mersenne Twister scheme has ~20,000 bits of internal state, and you cannot ask the user to provide 20,000 bits. There are sometimes ill-behaved initial states, but it is always the responsibility of the random number library to expand somehow the user-provided arbitrary seed into a well-behaved initial state.
