How to build a symmetric similarity model on top of embeddings?
That's an interesting question ! Surprisingly, I never read specific scientific papers on this particular problem. Here you want to learn a symmetric function that transforms two input vectors into a scalar. In a more general setting, we can find in the literature some needs for permutation invariant functions (see the Set Transformer and the Aggregation Schemes for Graph Neural Network ). Basically, you have two options:


As you pointed out, you can take whatever non symmetric function, such as concatenation + feed forward network and train it with (x,y) and (y,x) pairs. But there might be some smarter way...
Design a permutation invariant architecture. Here your chance is that you only have 2 embeddings $e_i$ and $e_j$. Sadly, you say they can't be trusted for their mutual dot product. Then, I might suggest to apply some linear/nonlinear transformation to your embeddings i.e. define a trainable matrix $Q$, and compute the dot product in the new space $v_i = Qe_i$ and $v_j = Qe_j$. By training $Q$ with example pairs $(e_i,e_j)$, you'll find a suitable transformation of the embeddings such that the dot product $v_i \cdot v_j$ fit the true similarity. You can of course use a more complex transformation (multiple non linear transformations for example). The key points is that using the same parameters for transforming both embeddings ensure the symmetry.


Bonus points: I'm not sure of your question, but my proposition in (2) might naturally deal with transitivity and reflexivity.