title: How to implement network using Bert as a paragraph encoder in long text classification, in keras?
body: <p>I am doing a long text classification task, which has more than 10000 words in doc, I am planing to use Bert as a paragraph encoder, then feed the embeddings of paragraph to BiLSTM step by step.
  The network is as below:</p>
  <blockquote>
  <p>Input: (batch_size, max_paragraph_len, max_tokens_per_para,embedding_size)</p>
  <p>bert layer: (max_paragraph_len,paragraph_embedding_size)</p>
  <p>lstm layer: ???</p>
  <p>output layer: (batch_size,classification_size)</p>
  </blockquote>
  <p>How to implement it with keras?
  I am using keras's load_trained_model_from_checkpoint to load bert model</p>
  <pre><code>bert_model = load_trained_model_from_checkpoint(
          config_path,
          model_path,
          training=False,
          use_adapter=True,
          trainable=['Encoder-{}-MultiHeadSelfAttention-Adapter'.format(i + 1) for i in range(layer_num)] +
              ['Encoder-{}-FeedForward-Adapter'.format(i + 1) for i in range(layer_num)] +
              ['Encoder-{}-MultiHeadSelfAttention-Norm'.format(i + 1) for i in range(layer_num)] +
              ['Encoder-{}-FeedForward-Norm'.format(i + 1) for i in range(layer_num)],
          )
  </code></pre>