title: Finetuning BERT on custom data
body: <p>I want to train a <strong>21 class</strong> text classification model using Bert. But I have very little training data, so a downloaded a similar dataset with <strong>5 classes</strong> with 2 million samples.t 
  And finetuned downloaded data with uncased pretrained model provided by bert.
  And got about 98% validation accuracy.
  Now, I want to use this model as pretrained model for my small custom data.
  But I am getting <code>shape mismatch with tensor output_bias from checkpoint reader</code> error as the check-point model has 5 classes and my custom data has 21 classes.</p>
  
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>NFO:tensorflow:Calling model_fn.
  INFO:tensorflow:Running train on CPU
  INFO:tensorflow:*** Features ***
  INFO:tensorflow:  name = input_ids, shape = (32, 128)
  INFO:tensorflow:  name = input_mask, shape = (32, 128)
  INFO:tensorflow:  name = is_real_example, shape = (32,)
  INFO:tensorflow:  name = label_ids, shape = (32, 21)
  INFO:tensorflow:  name = segment_ids, shape = (32, 128)
  Tensor("IteratorGetNext:3", shape=(32, 21), dtype=int32)
  WARNING:tensorflow:From /home/user/Spine_NLP/bert/modeling.py:358: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
  Instructions for updating:
  Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
  WARNING:tensorflow:From /home/user/Spine_NLP/bert/modeling.py:671: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
  Instructions for updating:
  Use keras.layers.dense instead.
  INFO:tensorflow:num_labels:21;logits:Tensor("loss/BiasAdd:0", shape=(32, 21), dtype=float32);labels:Tensor("loss/Cast:0", shape=(32, 21), dtype=float32)
  INFO:tensorflow:Error recorded from training_loop: Shape of variable output_bias:0 ((21,)) doesn't match with shape of tensor output_bias ([5]) from checkpoint reader.</code></pre>
  </div>
  </div>
  </p>