class_docstr: Abstract Configuration class to store `sparsity configuration of a self attention layer`.
  It contains shared property of different block-sparse sparsity patterns. However, each class needs to extend it based on required property and functionality.
docstr: Initialize the Sparsity Pattern Config.
  
  For usage example please see, TODO DeepSpeed Sparse Transformer Tutorial
  
  Arguments:
       num_heads: required: an integer determining number of attention heads of the layer.
       block: optional: an integer determining the block size. Current implementation of sparse self-attention is based on blocked sparse matrices. In which this parameter defines size of such blocks, `Block X Block`.
       different_layout_per_head: optional: a boolean determining if each head should be assigned a different sparsity layout; default is false and this will be satisfied based on availability.
docstr_tok: ["Initialize","the","Sparsity","Pattern","Config",".","For","usage","example","please","see",",","TODO","DeepSpeed","Sparse","Transformer","Tutorial","Arguments",":","num_heads",":","required",":","an","integer","determining","number","of","attention","heads","of","the","layer",".","block",":","optional",":","an","integer","determining","the","block","size",".","Current","implementation","of","sparse","self-attention","is","based","on","blocked","sparse","matrices",".","In","which","this","parameter","defines","size","of","such","blocks",",","`","Block","X","Block","`",".","different_layout_per_head",":","optional",":","a","boolean","determining","if","each","head","should","be","assigned","a","different","sparsity","layout",";","default","is","false","and","this","will","be","satisfied","based","on","availability","."]
code: def __init__(
      self, num_heads, block=16, different_layout_per_head=False
  ):
      """Initialize the Sparsity Pattern Config.
  
      For usage example please see, TODO DeepSpeed Sparse Transformer Tutorial
  
      Arguments:
           num_heads: required: an integer determining number of attention heads of the layer.
           block: optional: an integer determining the block size. Current implementation of sparse self-attention is based on blocked sparse matrices. In which this parameter defines size of such blocks, `Block X Block`.
           different_layout_per_head: optional: a boolean determining if each head should be assigned a different sparsity layout; default is false and this will be satisfied based on availability.
      """
  
      self.num_heads = num_heads
      self.block = block
      self.different_layout_per_head = different_layout_per_head
      self.num_layout_heads = (
          num_heads if different_layout_per_head else 1
      )
  
imports: import torch
  import random
label_desc: Use image node_id 1 to create a new SparsityConfig object  with example: obj = SparsityConfig(num_heads, block, different_layout_per_head)