PyTorch CUDA: "to(device)" vs "device=device" for tensors?
I found a somewhat similar question here What is the difference between model.to(device) and model=model.to(device)?, but I would like to check again if the same applies to my example:

Using .to(self.device)

mask = torch.tril(torch.ones(len_q, len_k)).type(torch.BoolTensor).to(self.device)`

and

Using device=self.device

mask = torch.tril(torch.ones((trg_len, trg_len), device = self.device)).bool()

Are they both accomplishing the same thing - ensuring that mask goes to the GPU?

The torch.Tensor.to function will make a copy of your tensor on the destination device. While setting the device option on initialization will place it there on init, so there is no copy involved.
So in your case you would rather do:
&gt;&gt;&gt; mask = torch.tril(torch.ones(len_q, len_k), device=self.device)

But to give an answer to your question, both have the effect of placing mask on self.device. The only difference is that in the former you will have a copy of your data on both devices.

The same can be said for torch.Tensor.bool vs. initializing with dtype:
&gt;&gt;&gt; torch.randint(0, 1, (10,)).bool()

Will make a copy, while the following won't:
&gt;&gt;&gt; torch.randint(0, 1, (10,), dtype=torch.bool)

However, torch.tril doesn't provide a dtype option, so it is not relevant here.
