title: How to add LSTM layer on top of Huggingface BERT model
body: <p>I am working on a binary classification task and would like to try adding lstm layer on top of the last hidden layer of huggingface BERT model, however, I couldn't reach the last hidden layer. Is it possible to combine BERT with LSTM?</p>
  <pre class="lang-py prettyprint-override"><code>tokenizer = BertTokenizer.from_pretrained(model_path)
  tain_inputs, train_labels, train_masks = data_prepare_BERT(
      train_file, lab2ind, tokenizer, content_col, label_col, 
      max_seq_length)
  validation_inputs, validation_labels, validation_masks = data_prepare_BERT(
      dev_file, lab2ind, tokenizer, content_col, label_col,max_seq_length)
  
  # Load BertForSequenceClassification, the pretrained BERT model with a single linear classification layer on top.
  model = BertForSequenceClassification.from_pretrained(
      model_path, num_labels=len(lab2ind))
  </code></pre>