Broadcasting semantics
Many PyTorch operations support NumPy's broadcasting semantics. See
https://numpy.org/doc/stable/user/basics.broadcasting.html for
details.
In short, if a PyTorch operation supports broadcast, then its Tensor
arguments can be automatically expanded to be of equal sizes (without
making copies of the data).
General semantics
Two tensors are "broadcastable" if the following rules hold:


Each tensor has at least one dimension.


When iterating over the dimension sizes, starting at the trailing
  dimension, the dimension sizes must either be equal, one of them is
  1, or one of them does not exist.


For Example:



x=torch.empty(5,7,3)
y=torch.empty(5,7,3)
   # same shapes are always broadcastable (i.e. the above rules always hold)
x=torch.empty((0,))
y=torch.empty(2,2)
   # x and y are not broadcastable, because x does not have at least 1 dimension



# can line up trailing dimensions



x=torch.empty(5,3,4,1)


