Best practice for short sentences in a deep learning network In a deep learning network (CNN or RNN), we might use word embeddings such as FastText, Glove, etc. to represent the input text. My question is:

If I'm working on a data from Twitter, and I have a variety of text lengths, as in the picture below (example):



sometime I have a few sentences of a length larger than 150 and the average length of the rest of the sentences is 48.

Here, what I noticed in some implementations online that they pad the short sentences with "PAD" word to increase their size to reach the length of the largest sentence (Red spaces in the picture), where this PAD word is filled with a random value. Is it a good practice to do so?

If my dataset contains 5000 tweets, and I have about 20 tweets with a length larger than 150 , the random value will affect the classification task with a high ratio since most of the sentences will be padded with a random value.
Yes, when you have different sequences with different length you can pad all the sequences so that they have all the length of the longest one. is a good practice and doesn t impact the learning mode. Another solution could be online learnig. With online learning you give to the model 1 sample at the time (batch_size =1) and in keras you can train the model with batch with different lengths. 

With online learning your computation is slower, but theoretically your model need less epochs to learn.

If you want to pad you should pad with all 0s and not random values.