title: Copy one layer's weights from one Huggingface BERT model to another
body: <p>I have a pre-trained model which I load like so:</p>
  <pre><code>from transformers import BertForSequenceClassification, AdamW, BertConfig, BertModel
  model = BertForSequenceClassification.from_pretrained(
      &quot;bert-base-uncased&quot;, # Use the 12-layer BERT model, with an uncased vocab.
      num_labels = 2, # The number of output labels--2 for binary classification.
                      # You can increase this for multi-class tasks.   
      output_attentions = False, # Whether the model returns attentions weights.
      output_hidden_states = False, # Whether the model returns all hidden-states.
  )
  </code></pre>
  <p>I want to create a new model with the same architecture, and random initial weights, <em>except</em> for the embedding layer:</p>
  <pre><code>==== Embedding Layer ====
  
  bert.embeddings.word_embeddings.weight                  (30522, 768)
  bert.embeddings.position_embeddings.weight                (512, 768)
  bert.embeddings.token_type_embeddings.weight                (2, 768)
  bert.embeddings.LayerNorm.weight                              (768,)
  bert.embeddings.LayerNorm.bias                                (768,)
  </code></pre>
  <p>It seems I can do this to create a new model with the same architecture, but then <em>all</em> the weights are random:</p>
  <pre><code>configuration   = model.config
  untrained_model = BertForSequenceClassification(configuration)
  </code></pre>
  <p>So how do I copy over <code>model</code>'s embedding layer weights to the new <code>untrained_model</code>?</p>