Continuous Bag Of Words (CBOW) network architecture?
What is the proper architecture to train CBOW encodings?

The original paper by Mikolov et al uses 1 hidden layer. However, for NLP tasks (and deep learning in general), there is no &quot;correct&quot; number of layers to use.
For some tasks, you might find using more hidden layers to be better, for other tasks maybe one hidden layer is sufficient. The number of layers is a hyperparameter, along with other 'design choices' like the dimension of the embedding vectors, the size of the vocabulary, and many, many more.

If this multiple hidden layer approach is correct, how do you not lose semantic information when you only use one of the layers as the encodings?

I think of it this way. When training CBOW, the hidden layers learn some 'relationship function' between the input context words and the output target word. In order for the 'relationship function' to perform well, the embeddings must be arranged accordingly (in N-dimensional space), and it just happens that the optimal arrangement encodes semantic information.
So the hidden layers shouldn't really have anything to do with semantic information. However, because of the immense modeling power of hidden layers (which make up neural networks), the truth is the embeddings will inevitably lose some semantic information to the hidden layers.

If the single hidden layer approach is correct, does anyone have examples of this being implemented using this approach in PyTorch (fine if no)?

Here's literally the 1st github example off Google.