title: how to get output from bert to lstm then classifer using keras?
body: <p>I want get output from bert embedding layer  to lstm then classifer using keras .how to extract this layer from outputs to use it as input for lstm layer</p>
  <pre><code># Create the BertClassfier class
  class BertClassifier(nn.Module):
      &quot;&quot;&quot;Bert Model for Classification Tasks.
      &quot;&quot;&quot;
      def __init__(self, freeze_bert=False):
          &quot;&quot;&quot;
          @param    bert: a BertModel object
          @param    classifier: a torch.nn.Module classifier
          @param    freeze_bert (bool): Set `False` to fine-tune the BERT model
          &quot;&quot;&quot;
          super(BertClassifier, self).__init__()
          # Specify hidden size of BERT, hidden size of our classifier, and number of labels
          D_in, H, D_out = 64,64, 6
  
          # Instantiate BERT model
          self.bert = BertModel.from_pretrained('bert-base-uncased')
  
          # Instantiate an one-layer feed-forward classifier
          self.classifier = nn.Sequential(
              nn.Linear(D_in, H),
              nn.ReLU(),
              #nn.Dropout(0.5),
              #nn.Linear(H, D_out)
          )
  
          # Freeze the BERT model
          if freeze_bert:
              for param in self.bert.parameters():
                  param.requires_grad = False
          
      def forward(self, input_ids, attention_mask):
          &quot;&quot;&quot;
          Feed input to BERT and the classifier to compute logits.
          @param    input_ids (torch.Tensor): an input tensor with shape (batch_size,
                        max_length)
          @param    attention_mask (torch.Tensor): a tensor that hold attention mask
                        information with shape (batch_size, max_length)
          @return   logits (torch.Tensor): an output tensor with shape (batch_size,
                        num_labels)
          &quot;&quot;&quot;
          # Feed input to BERT
          outputs = self.bert(input_ids=input_ids,
                              attention_mask=attention_mask)
          print(outputs)
          # Extract the last hidden state of the token `[CLS]` for classification task
          #last_hidden_state_cls = outputs[0][:, 0, :]
          #print( last_hidden_state_cls)
          # Feed input to classifier to compute logits
          #logits = self.classifier(last_hidden_state_cls)
          #print(logits)
  
          return   outputs
  </code></pre>
  <p>how to get output of hidden layers of bert to be input for lstm but i got error in bert_out</p>
  <p>outputs =BertClassifier( )</p>
  <p>bert_out = SpatialDropout1D(0.2)(outputs)</p>
  <p>l_lstm = LSTM(64)(bert_out)</p>
  <p>newout = keras.layers.Dense(64,activation='relu')(l_lstm)</p>
  <p>preds = keras.layers.Dense(units=7,activation='softmax')(newout)</p>
  <p>model = keras.models.Model([input_ids, attention_mask], preds)</p>
  <p>anyone can suggest any solution for this problem or different method to apply this model</p>