Evaluate document similarity / content-based recommender system I'm planning on building a basic content-based recommender system with word2vec and cosine similarity.
The data consists of 300k documents in varying length.
How do I evaluate my model if I have no labels / categories whatsoever?
When you don't have label/categories then it's called Unsupervised Learning. You can solve this problem via Latent Dirichlet Allocation (LDA) model and then evaluate your model by splitting the texts in half and compare the topic assignment for each half using cosine similarity. The more similar the topic assignment, the better.
Example