How would I implement a model in Keras to generate embeddings based on documents? I would like to create a model which generates embeddings for documents. I can create the model, but I was wondering how I would implement a training scheme where data consisted of two documents, either identical or different, and would be fed in parallel to the model (two instances running at the same time), which would create embeddings for their respective document. I want to use cosine distance to see similarity, and distance would be lower for documents that are the same. How would I train/implement this?
For starters, it depends on why you're trying to create Document Embeddings.
There's the TF-IDF algorithm; it's simple to grasp and implement, and it facilitates using cosine distance as a metric. To train in parallel, you could probably feed both documents to the model and have a synchronized counter doing the calculations such as Term Frequency and Inverse Document Frequency, and then in the end you can use those values to scale the elements of your TFIDF matrix.
Alternatively, you can use the Doc2Vec algorithm as explained here.
It's an extension of the Word2Vec model, but parallelizing the training for this one might be more complicated.
I'm sure there are more ways, but these are the two most popular that I know of.