Best approach to create a text classification model with two inputs? I'm looking to train a model with two text inputs (sentences) and a binary classification. Essentially, for 2 given sentences, are they paraphrases or not.
I want to use the Microsoft research Paraphrase Detection dataset as I feel this would be a good place to start.
What would be the best approach to do this?
Thanks in advance, and sorry for my naïvete.
All current pre-trained Transformers (BERT, RoBERTa, ...) can take two sentences as an input, simply by concatenating the sentences and separating them by a special token. You can then finetune the model on the paraphrase dataset.
The Microsoft Research Parallel Corpus is part of the GLUE benchmark for sentence representation evaluation. It has a public leaderboard, so you can have a look and get an idea, how the best models look like.