class_docstr: After this was developed pytorch added their own implementation:
  https://pytorch.org/docs/master/nn.html#multiheadattention
class_docstr_tok: ["After","this","was","developed","pytorch","added","their","own","implementation",":","https",":","\/\/pytorch.org\/docs\/master\/nn.html","#","multiheadattention"]
code: def split_heads(self, x, k=False):
      new_x_shape = (
          *x.size()[:-1],
          self.heads_per_shard,
          x.size(-1) // self.heads_per_shard,
      )
      x = x.view(*new_x_shape)  # in Tensorflow implem: fct split_states
      if k:
          return x.permute(0, 2, 3, 1)
      else:
          return x.permute(0, 2, 1, 3)
  
file_path: summarize-from-feedback/summarize_from_feedback/models/attention.py
imports: from typing import Tuple
  import numpy
  import torch
  import torch.nn
  from summarize_from_feedback.models.ops import Conv1D
  from summarize_from_feedback.utils import exact_div
  from summarize_from_feedback.utils.dist_utils import Comm
label_desc: Use image node_id 4 for calling the AttentionFunc obj's underlying member method code with example usage: obj.split_heads(x, k) and returns: x, x