Pytorch Concatenate rows in alternate order
I am trying to code up the positional encoding in the transformers paper. In order to do so I need to do an operation similar to the following:

a = torch.arange(20).reshape(4,5) 
b = a * 2
c = torch.cat([torch.stack([a_row,b_row]) for a_row, b_row in zip(a,b)])


I feel like there might be a faster way to do the above? perhaps by adding a dimension on to a and b?

So turns out simple concatenation and reshaping does the trick:

c = torch.cat([a, b], dim=-1).view(-1, a.shape[-1])


When I timed it with the following it was about 2.3x faster than @dennlinger's answer:

improved2 = timeit.timeit("c = torch.cat([a, b], dim=-1).view(-1, a.shape[-1])",  
                          setup="import torch; from __main__ import a, b", 
                          number=10000) 
print(improved2/10000) 
# 7.253780400003507e-06
print(improved / improved2)
# 2.3988091506044955

