class_docstr: BERT model ("Bidirectional Encoder Representations from Transformers").
class_docstr_tok: ["BERT","model","(","``","Bidirectional","Encoder","Representations","from","Transformers","''",")","."]
docstr: Dimensionality of attention queries/keys/values.
code: def size_per_head_dim(self):
      """Dimensionality of attention queries/keys/values."""
      if self.config.attention_head_size:
          attention_head_size = self.config.attention_head_size
      else:
          if self.model_dim.size % self.num_heads_dim.size != 0:
              raise ValueError(
                  "The hidden size (%d) is not a multiple of the number of attention "
                  "heads (%d)"
                  % (self.model_dim.size, self.num_heads_dim.size)
              )
          attention_head_size = int(
              self.model_dim.size / self.num_heads_dim.size
          )
      return mtf.Dimension("attention_head", attention_head_size)
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import copy
  import json
  import math
  import re
  import mesh_tensorflow
  import mesh_tensorflow.transformer.moe
  import six
  import tensorflow.compat.v1
label_desc: Use image node_id 26 for calling the BertModel obj's underlying member method code with example usage: obj.size_per_head_dim() and returns: mtf