How can we compare, in terms of similarity, two pieces of text?
There are more than 1 way of doing this:


You can compute the bleu score between them if you are looking at the quality of machine translation. Check this link.
You can convert them into 2 vectors using doc2vec and find the similarity between the vectors using cosine similarity.
Siamese networks are something similar to what you are asking. They are neural nets that use distance metric for learning rather than a loss metric.


I don't understand why do you want to use neural network for comparing two text pieces. Generally comparisons are done by some of the distance metrics not by neural network.