How to find the mean and the covariance of a 2d activation map (pytorch)
I have a tensor of shape [h, w], which consists of a normalized, 2-dimensional activation map. Considering this to be some distribution, I want to find the mean and the covariance within this activation map in pytorch. Is there an efficient way to do that?

You can use the following code, where activation_map is a tensor of shape (h,w), with non-negative elements, and is normalised (activation_map.sum() is 1):
activation_map = torch.tensor(
    [[0.2, 0.1, 0.0],
     [0.1, 0.2, 0.4]])
h, w = activation_map.shape

range_h = torch.arange(h)
range_w = torch.arange(w)
idxs = torch.stack([
  range_w[None].repeat(h, 1),
  range_h[:, None].repeat(1, w)
  ])
map_flat = activation_map.view(-1)
idxs_flat = idxs.reshape(2, -1).T
mean = (map_flat[:, None] * idxs_flat).sum(0)
mats = idxs_flat[:, :, None] @ idxs_flat[:, None, :]
second_moments = (map_flat[:, None, None] * mats).sum(0)
covariance = second_moments - mean[:, None] @ mean[None]

# mean:
# tensor([1.1000, 0.7000])
# covariance:
# tensor([[0.6900, 0.2300],
#         [0.2300, 0.2100]])

