How does Continuous Bag of Words ensure that similar words are encoded as similar embeddings?
Unlike in skip-gram, the reason similar words have similar embeddings in CBOW is because the words show up in the same contexts of other skipped words.

lets assume two words $e_i$ and $e_j$ pop up in the exact same context of some word $e_k$ with 3 other context words as well. An example would be:


He leaped over the truck
He jumped over the truck


Where the italics represent the words with similar meanings but the bolded words above are the ones being predicted/skipped in CBOW. Lets now show this.

Let the rest of the context be denoted $\{e_r\}_r$ and the skipped word as $e_s$ and so the loss will try to minimize both $-e_s^T log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_i + \sum_re_r)]))$ and $-e_s^T log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_j + \sum_re_r)]))$.   

Assuming a long run withlarge enough batch-size (so ignoring catastrophic forgetting) we can essentially say it will be minimizing
$$-e_s^T [log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_i + \sum_re_r)])) +\\ log(softmax(\mathcal{U}\mathcal{V}[\frac{1}{R+1}(e_j + \sum_re_r)]))]$$
Now let $\mathcal{U}\mathcal{V}\frac{1}{R+1}\sum_re_r$ be denoted as $c$, $\mathcal{U}\mathcal{V}\frac{1}{R+1}e_i$ as $\hat u_i$, and $\mathcal{U}\mathcal{V}\frac{1}{R+1}e_j$ as $\hat u_j$. So we have to minimize
$$-e_s^T[log(softmax(\hat u_i+c)) + log(softmax(\hat u_j+c))]$$
or equivalently
$$-e_s^T\ log(softmax(\hat u_i+c) * softmax(\hat u_j+c))$$
As we know in crossentropy, we get our critical point when the inside of the log here would equal $e_s$, which is only possible if both $softmax(\hat u_i+c)$ and $softmax(\hat u_j+c)$ were to be equal to $e_s$. softmax is non-invertible and is invariant under addition only, therefore to achieve this we would need $\hat u_i + c$ to equal $\hat u_j + c + K$ where $K$ is just some arbitrary constant vector ($K=[k,k,k,...]$. Subtracting $c$ and multiplying by $R+1$ we make our model want  

$$\begin{align}
u_i &amp;= u_j + K \\
\implies K &amp;= \mathcal{U}\mathcal{V}(e_i -e_j) \\
\end{align}$$

This would mean for all common words, we see this activity that $u_{i-j}$ would be a constant, but this is difficult because that means a $\delta$ must exist such that $\mathcal{U}\delta = k*\vec 1$ and that all similar word pairs indexed by $(a,b)$ would have $v_a \approx v_b + const*\delta$ (because $u_a = u_b + K \implies v_b = v_a + \mathcal{U}^{-1}K$ and we denote $\delta = \mathcal{U}^{-1}K$). This constraint is heavy, tough to learn, and would also be difficult to process on multiple word senses that would appear in the vocabulary. This is because if it were to learn this, it enforces constraints on the images of $\mathcal{U}$ and $\mathcal{V}$ along with creating word vectors with highly different magnitudes making the learning process difficult. This would indicate that this constant would be low in practice. Therefore we would get $v_i \approx v_j$.