class_docstr: Initialize the Lightseq Transformer Decoder Layer.
  
  Static variable:
      layer_id: The layer-index counter starting from 0 and incrementing by 1 every time a layer object is instantiated,
      e.g. if a model has 24 transformer layers, layer_id goes from 0 to 23.
  Arguments:
      config: An object of LSTransformerDecoderLayer config, see get_config
  
      initial_weights: Optional: Only used for unit test
  
      initial_biases: Optional: Only used for unit test
class_docstr_tok: ["Initialize","the","Lightseq","Transformer","Decoder","Layer",".","Static","variable",":","layer_id",":","The","layer-index","counter","starting","from","0","and","incrementing","by","1","every","time","a","layer","object","is","instantiated",",","e.g",".","if","a","model","has","24","transformer","layers",",","layer_id","goes","from","0","to","23",".","Arguments",":","config",":","An","object","of","LSTransformerDecoderLayer","config",",","see","get_config","initial_weights",":","Optional",":","Only","used","for","unit","test","initial_biases",":","Optional",":","Only","used","for","unit","test"]
docstr: Returns the offset of each module's parameters among all
  parameters of a layer
docstr_tok: ["Returns","the","offset","of","each","module","'s","parameters","among","all","parameters","of","a","layer"]
code: def gen_offset(hidden_size, intermediate_size, nlayer):
      """Returns the offset of each module's parameters among all
      parameters of a layer
      """
      hs, ims = hidden_size, intermediate_size
      sizes = [
          hs * hs * 3,  # attn_qkvw
          hs * 3,  # attn_qkvb
          hs * hs,  # attn_ow
          hs,  # attn_ob
          hs,  # attn_nw
          hs,  # attn_nb
          hs * hs,  # encdec_attn_qw
          hs,  # encdec_attn_qb
          hs * hs,  # encdec_attn_ow
          hs,  # encdec_attn_ob
          hs,  # encdec_attn_nw
          hs,  # encdec_attn_nb
          hs * ims,  # inter_w
          ims,  # inter_b
          hs * ims,  # output_w
          hs,  # output_b
          hs,  # ffn_nw
          hs,  # ffn_nb
          24,
          hs * hs * 2 * nlayer,  # encdec_attn_kvw
          hs * 2 * nlayer,  # encdec_attn_kvb
      ]
      offsets = calc_offset(sizes)
      return offsets
  
imports: import __init__
  from itertools import zip_longest
  import copy
  import torch
  from torch import nn
  import math
  from dataclasses import dataclass
  from csrc.pytorch.layer_base import TransformerDecoderLayerBase
  from csrc.pytorch.util import copy_para, state_dict, calc_offset
  from csrc.pytorch.torch_transformer_layers import act_quant_config, weight_quant_config
  from csrc.pytorch.builder.cuda_layer_builder import CudaLayerBuilder
label_desc: Use image node_id 2 for calling the LSTransformerDecoderLayer obj's underlying member method code with example usage: obj.gen_offset(hidden_size, intermediate_size, nlayer) and returns: offsets