Text similarity using RNN Data set contains records of short text, typically a sentence. The goal is to find duplicated records and similar records. Currently, I have tried R package 'text2vec', the glove word vectors and the similarity APIs provided by the package.

There is a smaller subset of this data which is already tagged as duplicated. Currently, I have not factored in this as part of model training. Also, using the text2vec package, the results are not great on this test set. So now I am considering RNNs which are known to perform well in text similarity. 

Now, I need help in feature engineering and preparing my input layer. Sentences S1 and S2 that need to be compared differ in length (the word representations differ in their dimensions). How to normalize this difference? Should I consider bag-of-words or glove word vectors (word vectors being much superior representations)? Any inputs in this regard will help.
Doc2Vec, Mikolov's paper will solve your problem. Here is the paper. You can find a gensim implementationhere. While using RNN, using GLOVE or Googl Word2Vec will be always useful even if your sequences are of the same length. To solve the variable sequence problem you can pad the sentences and use bucketing or just pad to one uniform length, truncating longer sentences and padding shorter sentences.