How to handle Memory issues in training Word Embeddings on Large Datasets?
I'm not familiar with Torch, but since basically word2vec and doc2vec are considered, These models learn from each sentences and so there is no need to have all the sentences in the memory. You could iterate via each sentence in the corpora and let the model learn from each of the sentence. And that is probably how people train on huge corpora with or without high computation machines.

A short example in python:

class SentenceIterator(object):
   def __iter__(file_name):
      for line in open(file_name)
         yield line

sentences = SentenceIterator("path/to/file")
for line in sentences:
    model.train(line)


In this way, The memory is loaded with only one sentence at a time and when it is done the memory loads the next one. 
For building the vocabulary, you can do the whole iterating via all the documents to build the vocab first and then train the data, depending upon the word-embedding functions implemented.