


batched_dot(x, y) # tensor of size [2, 3]



If the inputs are not batched along the first dimension, "in_dims"
   specifies the dimension that each inputs are batched along as



torch.dot                            # [N], [N] -> []
batched_dot = torch.vmap(torch.dot, in_dims=1)  # [N, D], [N, D] -> [D]
x, y = torch.randn(2, 5), torch.randn(2, 5)
batched_dot(x, y)   # output is [5] instead of [2] if batched along the 0th dimension



If there are multiple inputs each of which is batched along
   different dimensions, "in_dims" must be a tuple with the batch
   dimension for each input as



torch.dot                            # [D], [D] -> []
batched_dot = torch.vmap(torch.dot, in_dims=(0, None))  # [N, D], [D] -> [N]
x, y = torch.randn(2, 5), torch.randn(5)
batched_dot(x, y) # second arg doesn't have a batch dim because in_dim[1] was None



If the input is a Python struct, "in_dims" must be a tuple