How to get padding mask from input ids?
Considering a batch of 4 pre-processed sentences (tokenization, numericalizing and padding) shown below:

batch = torch.tensor([
                      [1, 2, 0, 0],
                      [4, 0, 0, 0],
                      [3, 5, 6, 7]
])


where 0 states for [PAD] token.

Thus, what would be an efficient approach to generate a padding masking tensor of the same shape as the batch assigning zero at [PAD] positions and assigning one to other input data (sentence tokens)?

In the example above it would be something like:

padding_masking= 
tensor([
                      [1, 1, 0, 0],
                      [1, 0, 0, 0],
                      [1, 1, 1, 1]
])


You can get your desired result with 

padding_masking = batch &gt; 0


If you want ints instead of booleans, use

padding_masking.type(torch.int)

