Full Doc2Vec Implementation/Repdoduction in TensorFlow?
Implementations
These are all full scripts using Tensorflow, but just using Tensorflow does not ensure quality.
Bad
WangZ's implementation looks complete from a brief glance, although it only offers Adagrad and Stochastic Gradient Descent, two of the least reliable gradient descents. You really want minibatch gradient descent, which will predictably converge close to the local minimum where you initialize your system (think: optimization problems from calculus class). Beyond that, there's nothing really notable about his or her implementation.
Better
Jonathan's implementation is a bit better than WangZ's but worse than the Machine Learning Cookbook solution. He also uses Gensim instead of TensorFlow, which is not bad except from an engineering standpoint (it doesn't scale quite as well). He uses mini-batch gradient descent, but with hierarchical softmax instead of NCE. This is not going to make a huge difference except perhaps increasing training time.
Best
The Machine Learning Cookbook implementation is really just incredible. It follows all of the recommendations from Google's original papers on the topic, even improving on the gradient descent (this uses mini-batch gradient descent). Notably, this also uses NCE (Noise Contrastive Estimation), which I don't typically see, instead of hierarchical softmax to obtain the error for gradient descent, which Google showed produces slightly better results and works faster. A lot of thought clearly went into this, and whoever was behind it knew what s(he) was doing. I would go with this one if I were you.
PV-DM vs PV-DBOW
PV stands for Paragraph Vectors, the model you are talking about. It is also know as doc2vec. It comes in two flavors, CBOW (Continuous Bag of Words), sometimes also called DBOW (Distributed Bag of Words), and DM (Distributed Memory).
These are both distributed models in that each &quot;neuron&quot; contributes to the training task, but nothing has any meaning without the other &quot;neurons&quot; to give it context.
To paraphrase Google's documentation: The CBOW/DBOW model predicts the target word from a bag of context words (e.g. &quot;mat&quot; from &quot;the cat sits on the&quot;) while the DM model predicts a skip-gram from a single word (e.g. &quot;sits .. mat&quot; from &quot;cat&quot;).
In practice, research shows that the skip-gram/DM model works better if you have a lot of data and the CBOW/DBOW model works better with less. That said, this technique (and deep learning in general) isn't particularly effective without lots of training data.
IMDB
Any of these implementations can be trained on comment data from IMDB. For example, the Machine Learning Cookbook implementation is hard-coded to pull data from a similar dataset . To change where it gets the data from, you would modify the load_movie_data() function in text_helpers.py (same directory in the repository as your link, but a different file) to pull from your preferred source. For instance, you might want to modify it to pull data from the local file-system instead.