How to introduce a loss to get two matrices similar
I am training a neural network and I want two matrices to be similar (the covariance matrices). My naive approach was to use a loss based on the difference, such as the L1 loss. But this also forced the matrices to become small, which is not what I want.
Does anyone have an idea here?
Thanks a lot!

There are many metrics you can use (euclidian distances, cosine similarity, the Bhattacharyya similarity for non-negative features, the Jensen-Shannon divergence).
The cosine similarity seems like a good place to start. You can achieve this by considering both n x m matrices in a n*m dimensional space. And, compare those two vectors with the cosine similarity.
In practice, this can be done using torch.flatten and torch.nn.functionnal.cosine_similarity. Or equivalently with a nn.Flatten layer and nn.CosineSimilarity.
Here I've taken the functional route:
&gt;&gt;&gt; x = torch.rand(1, 10, 10)
&gt;&gt;&gt; y = torch.rand(1, 10, 10)

&gt;&gt;&gt; F.cosine_similarity(torch.flatten(x, 1), torch.flatten(y, 1))
tensor([0.6220])

Notice you will need an extra dimension for the batch: axis=0.

Edit - if you're not working with batches, you could simply broadcast both tensors to 1D tensors:
&gt;&gt;&gt; F.cosine_similarity(x.reshape(1, -1), y.reshape(1, -1))

