class_docstr: Initialize the Lightseq Transformer Decoder Layer.
  
  Static variable:
      layer_id: The layer-index counter starting from 0 and incrementing by 1 every time a layer object is instantiated,
      e.g. if a model has 24 transformer layers, layer_id goes from 0 to 23.
  Arguments:
      config: An object of LSTransformerDecoderLayer config, see get_config
  
      initial_weights: Optional: Only used for unit test
  
      initial_biases: Optional: Only used for unit test
code: def __init__(self, config, initial_weights=None, initial_biases=None):
      super(LSTransformerDecoderLayer, self).__init__()
  
      self.config = copy.deepcopy(config)
      self.config.layer_id = LSTransformerDecoderLayer.layer_id
      LSTransformerDecoderLayer.layer_id = (
          LSTransformerDecoderLayer.layer_id + 1
      )
  
      print("Lightseq Transformer config is ", self.config.__dict__)
  
      self.quant_mode = False
  
      if self.config.local_rank >= 0:
          torch.cuda.set_device(self.config.local_rank)
  
      # create the layer in cuda kernels.
      cuda_module = transformer_cuda_module
      create_layer_func = (
          cuda_module.create_transformer_decoder_layer_fp16
          if self.config.fp16
          else cuda_module.create_transformer_decoder_layer_fp32
      )
  
      create_layer_func(
          self.config.layer_id,
          self.config.max_batch_tokens,
          self.config.max_seq_len,
          self.config.hidden_size,
          self.config.nhead,
          self.config.intermediate_size,
          self.config.attn_prob_dropout_ratio,
          self.config.activation_dropout_ratio,
          self.config.hidden_dropout_ratio,
          self.config.pre_layer_norm,
          self.config.activation_fn,
      )
  
      hs = self.config.hidden_size
      ims = self.config.intermediate_size
      self.hs = hs
      self.ims = ims
  
      self.para_offset = LSTransformerDecoderLayer.gen_offset(
          hs, ims, self.config.nlayer
      )
      if self.config.layer_id != 0:
          self.para_offset = self.para_offset[:-2]
      self.para = nn.Parameter(torch.Tensor(self.para_offset[-1]))
  
      if initial_weights is None or initial_biases is None:
          # enc-dec kv weights and bias
          self.init_transformer_weights()
          return
  
      # For testing only.
      attn_qkvw = [ele.detach().clone() for ele in initial_weights[:3]]
      attn_qkvw = torch.cat(attn_qkvw, dim=0)
      weights = [attn_qkvw] + [
          copy_para(ele) if ele is not None else None
          for ele in initial_weights[3:]
      ]
  
      attn_qkvb = [ele.detach().clone() for ele in initial_biases[:3]]
      attn_qkvb = torch.cat(attn_qkvb, dim=0)
      biases = [attn_qkvb] + [
          copy_para(ele) if ele is not None else None
          for ele in initial_biases[3:]
      ]
  
      idx = 0
      for w, b in zip_longest(weights, biases):
          if w is not None:
              cur_para = self._get_weights(idx)
              assert cur_para.numel() == w.numel()
              cur_para.copy_(w.view(-1))
              idx += 1
  
          if b is not None:
              cur_para = self._get_weights(idx)
              assert cur_para.numel() == b.numel()
              cur_para.copy_(b.view(-1))
              idx += 1
  
imports: from itertools import zip_longest
  import math
  from dataclasses import dataclass
  import copy
  import torch
  from torch import nn
  from torch.autograd import Function
  from lightseq.training.ops.pytorch import TransformerBuilder
  from lightseq.training.ops.pytorch.builder import TransformerBuilder
  from lightseq.training.ops.pytorch.util import copy_para, state_dict, MODEL_ARCH, check_config, calc_offset
  from lightseq.training.ops.pytorch.quantization import weight_quant_config, act_quant_config, relu_quant_config
  from lightseq.training.ops.pytorch.layer_base import TransformerDecoderLayerBase
label_desc: Use image node_id 1 to create a new LSTransformerDecoderLayer object from inherited base classes: TransformerDecoderLayerBase with example: obj = LSTransformerDecoderLayer(config, initial_weights, initial_biases)