Does anybody know where this rule of thumb came from? Rule of them: embedding vector dimension should be the 4th root of the number of categories
google has published a word embedding that has embedding dimension of 300. Following the rule you have given it should have trained on $300^4 = 8.1*10^9$ words. If google is using ngrams instead of just words, then it seems plausible.