TLDR Bot - Sentence Tagging w/ BERT Currently making a bot that condenses news articles. I'm tagging sentences as important or not important using a simple BERT classifier. The results were... not great. I'm really interested in how I can improve the results using LSTM.

I'm now batching 5 sentences together, calculating their BERT encodings, and then using 2 LSTM Layers, one backwards one forwards, to predict if the sentence is important.

Unfortunately I'm now calculating 5 times the number of embeddings, and if it doesn't work, I can't seem to figure out how to feed a variable number of things into BERT using Tensorflow, to see if I can tweak some results.

Are there other methods to add surrounding sentences to this context?
You're trying to do Extractive Text Summarization.

You're first approach is the right one, using BERT + CLassifier. 

From NLP Progress, the best method so far is Presumm.