Question about BERT embeddings with high cosine similarity
It's really hard to say if the BERT assigns similar embeddings to the same words in the similar context. Most likely it will not. This is because, the embeddings, is not just function of context (other words in the sentence), but the position as well. Hence, even if the words occurs in similar context, with different positions, they can have different embeddings.