Best practices for generating a random seeds to seed Pytorch?
What I really want is to seed the dataset and dataloader. I am adapting code from:

https://gist.github.com/kevinzakka/d33bf8d6c7f06a9d8c76d97a7879f5cb

Anyone know how to seed this properly? What are the best practices for seeding things in Pytorch.

Honestly, I have no idea if there is an algorithm specific way for GPU vs CPU. I care mostly about general pytorch and make sure my code is "truly random". Specially when it uses GPU I guess...



related: 


https://discuss.pytorch.org/t/best-practices-for-seeding-random-numbers-on-gpu/18751
https://discuss.pytorch.org/t/the-random-seed/19516/4
https://discuss.pytorch.org/t/best-practices-for-generating-a-random-seed-to-seed-pytorch/52894/2




My answer was deleted and here is its content:

I don't know if this is the best for pytorch but this is what seems the best for any programming language:



Usually the best random sample you could get in any programming language is generated through the operating system. In Python, you can use the os module:

random_data = os.urandom(4)

In this way you get a cryptographic safe random byte sequence which you may convert in a numeric data type for using as a seed.

seed = int.from_bytes(random_data, byteorder="big")

EDIT: the snippets of code works only on Python 3



'''
Greater than 4 I get this error:

ValueError: Seed must be between 0 and 2**32 - 1
'''

RAND_SIZE = 4

Have a look at https://pytorch.org/docs/stable/notes/randomness.html

This is what I use

def seed_everything(seed=42):
  random.seed(seed)
  os.environ['PYTHONHASHSEED'] = str(seed)
  np.random.seed(seed)
  torch.manual_seed(seed)
  torch.backends.cudnn.deterministic = True
  torch.backends.cudnn.benchmark = False


the two last parameters (cudnn) are for GPU
