How to make a PyTorch Distribution on GPU
Is it possible to make the PyTorch distributions create their samples directly on GPU.

If I do 

from torch.distributions import Uniform, Normal
normal = Normal(3, 1)
sample = normal.sample()


Then sample will be on CPU. Of course it is possible to do sample = sample.to(torch.device("cuda")) to make it on GPU. But is there a way to have the sample go directly to GPU without first creating it on CPU?

PyTorch distributions inherit from Object, not nn.Module so it does not have a to method the put the distribution instance on GPU. 

Any ideas?

Distributions use the reparametrization trick. Thus giving size 0 tensors which are on GPU to the distribution constructor works. As follows:

normal = Normal(torch.tensor(0).to(device=torch.device("cuda")), torch.tensor(1).to(device=torch.device("cuda")))

