title: what is the difference between pooled output and sequence output in bert layer?
body: <p>everyone! I was reading about Bert and wanted to do <strong>text classification</strong> with its word embeddings. I came across this line of code:</p>
  <pre><code>pooled_output, sequence_output = self.bert_layer([input_word_ids, input_mask, segment_ids])   
  </code></pre>
  <p>and then:</p>
  <pre><code>clf_output = sequence_output[:, 0, :]
  out = Dense(1, activation='sigmoid')(clf_output)
  </code></pre>
  <p>But I can't understand the use of pooled output. <strong>Doesn't sequence output contain all the information including the word embedding of ['CLS']?</strong> If so, why do we have pooled output?</p>
  <p><em>Thanks in advance!</em></p>